Week 1 - Introduction to Physical AI
Learning Objectives
By the end of this week, you will be able to:
- Define Physical AI and understand its core components
- Identify the hardware requirements for Physical AI development
- Understand the curriculum structure and learning path
- Set up your development environment
Overview
Physical AI represents the convergence of artificial intelligence with physical systems, enabling robots to perceive, reason, and act in the real world. This course focuses on building humanoid robots with conversational AI capabilities using cutting-edge tools and frameworks.
What is Physical AI?
Physical AI encompasses:
- Perception: Computer vision, sensor fusion, and environmental understanding
- Reasoning: Decision-making, planning, and language understanding
- Action: Motor control, manipulation, and locomotion
- Learning: Adaptation through experience and imitation
Hardware Requirements
Minimum Specifications
| Component | Requirement | Purpose |
|---|---|---|
| GPU | NVIDIA RTX 3060 (12GB VRAM) | Isaac Sim, training, simulation |
| CPU | Intel i7 / AMD Ryzen 7 (8 cores) | ROS 2 nodes, simulation |
| RAM | 32GB DDR4 | Multi-process simulation |
| Storage | 500GB NVMe SSD | Dataset storage, fast I/O |
| OS | Ubuntu 22.04 LTS | ROS 2 Humble compatibility |
Recommended Specifications
| Component | Requirement | Purpose |
|---|---|---|
| GPU | NVIDIA RTX 4090 (24GB VRAM) | Large model training, complex scenes |
| CPU | Intel i9 / AMD Ryzen 9 (16 cores) | Parallel simulation, real-time control |
| RAM | 64GB DDR5 | Multi-robot simulation |
| Storage | 1TB NVMe SSD | Large datasets, fast checkpointing |
Edge Deployment Hardware
| Component | Requirement | Purpose |
|---|---|---|
| Edge AI | NVIDIA Jetson Orin Nano / Orin NX | On-robot inference |
| Sensors | RGB-D cameras, IMU, LiDAR | Perception stack |
| Actuators | Dynamixel servos, BLDC motors | Robot actuation |
Humanoid Robotics Landscape
Evolution of Humanoid Robots
| Era | Robot | Organization | Key Milestone |
|---|---|---|---|
| 1973 | WABOT-1 | Waseda University | First full-scale humanoid — rule-based walking |
| 2000 | ASIMO | Honda | Dynamic bipedal walking, stair climbing |
| 2013 | Atlas | Boston Dynamics | Hydraulic, outdoor terrain navigation |
| 2016 | NAO / Pepper | SoftBank Robotics | Social robotics, retail deployment |
| 2022 | Optimus (Gen 1) | Tesla | AI-driven, trained on human video data |
| 2023 | H1 / G1 | Unitree Robotics | Affordable, open SDK, ROS 2 compatible |
| 2024 | Figure 01 / 02 | Figure AI | OpenAI-powered reasoning + dexterous hands |
Why Humanoid Form Factor?
Humanoids are built to operate in human-designed environments — they can use the same doors, stairs, tools, and workspaces as people. This makes them uniquely suited for factory floors, disaster response, and home assistance without requiring infrastructure changes.
Embodied Intelligence
Unlike a chatbot (pure digital AI), an embodied agent must:
- Perceive the physical world through sensors
- Maintain a real-time model of its body position (proprioception)
- Act under strict latency constraints (motor commands must arrive in milliseconds)
- Handle physical uncertainty — a robot cannot "undo" a dropped object
This is what separates Physical AI from all previous AI domains.
Sensor Systems
Every humanoid robot is built around four core sensor types:
1. LiDAR (Light Detection and Ranging)
Fires laser pulses and measures return time to build a 3D point cloud of the environment.
- Range: 0.1m – 100m
- Use case: Obstacle detection, SLAM (Simultaneous Localization and Mapping)
- Common hardware: Ouster OS1-16, Velodyne VLP-16
2. RGB-D Cameras (Depth Cameras)
Combines a standard color camera with a depth sensor (structured light or Time-of-Flight).
- Output: Color image + per-pixel depth map
- Use case: Object detection, grasping, scene understanding
- Common hardware: Intel RealSense D435i (includes built-in IMU)
3. IMU (Inertial Measurement Unit)
Measures linear acceleration (accelerometer) + angular velocity (gyroscope).
- Critical role: Balance control — the robot must know if it is tipping before it falls
- Fusion: Combined with joint encoders for full proprioceptive awareness
- Common hardware: BNO055 (9-axis), built into most Jetson dev kits
4. Force/Torque Sensors
Measure the forces and torques at joints or end-effectors (hands/feet).
- Use case: Safe human interaction, compliant grasping, terrain adaptation
- Key insight: A robot stepping on uneven ground must feel the force difference between feet to adjust balance — cameras alone are too slow for this
Sensor Fusion Architecture
LiDAR ──────┐
RGB-D ──────┤──► Sensor Fusion Node ──► World Model ──► Planning
IMU ────────┤ (EKF / UKF)
F/T ────────┘
Development Environment Setup
Software Stack
# Ubuntu 22.04 LTS prerequisites
sudo apt update && sudo apt upgrade -y
# Install ROS 2 Humble
sudo apt install software-properties-common
sudo add-apt-repository universe
sudo apt update && sudo apt install curl -y
curl -sSL https://raw.githubusercontent.com/ros/rosdistro/master/ros.key | sudo apt-key add -
echo "deb http://packages.ros.org/ros2/ubuntu jammy main" | sudo tee /etc/apt/sources.list.d/ros2.list
sudo apt update && sudo apt install ros-humble-desktop -y
# Install NVIDIA drivers (for RTX GPUs)
sudo apt install nvidia-driver-535 -y
# Install CUDA Toolkit
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt update && sudo apt install cuda-12-4 -y
Resources
Documentation
Videos
Reading
- "Probabilistic Robotics" by Thrun, Burgard, and Fox
- "Modern Robotics" by Lynch and Park
Exercises
- Environment Verification: Verify your development environment meets the minimum requirements
- ROS 2 Installation: Complete ROS 2 Humble installation and run
ros2 run demo_nodes_cpp talker - GPU Verification: Run
nvidia-smito confirm GPU detection and driver version
Next Steps
Proceed to Week 2 - Physical AI Architecture to understand the system architecture.