Week 8 - Introduction to Isaac Sim

Learning Objectives

By the end of this week, you will be able to:

• Understand NVIDIA Isaac Sim architecture and capabilities
• Install and configure Isaac Sim with Omniverse
• Create basic simulation scenes using USD
• Integrate Isaac Sim with ROS 2

Overview

NVIDIA Isaac Sim is a GPU-accelerated robotics simulation platform built on NVIDIA Omniverse. It provides photorealistic rendering, accurate physics simulation, and synthetic data generation for training AI models.

Key Features

Feature	Description
Photorealistic Rendering	RTX-based ray tracing for realistic visuals
Physics Simulation	PhysX 5 for accurate rigid body dynamics
Synthetic Data	Domain randomization for training perception models
ROS 2 Integration	Native ROS 2 bridge for seamless integration
Reinforcement Learning	Isaac Gym for parallel RL training

Hardware Requirements

Minimum Specifications

Component	Requirement	Purpose
GPU	NVIDIA RTX 3060 (12GB VRAM)	RTX rendering, physics simulation
CPU	Intel i7 / AMD Ryzen 7 (8 cores)	Simulation orchestration
RAM	32GB DDR4	Scene loading, data processing
Storage	100GB NVMe SSD	Asset storage, caching
OS	Ubuntu 20.04/22.04 LTS	Isaac Sim compatibility

Recommended Specifications

Component	Requirement	Purpose
GPU	NVIDIA RTX 4090 (24GB VRAM)	Complex scenes, large models
CPU	Intel i9 / AMD Ryzen 9 (16 cores)	Multi-robot simulation
RAM	64GB DDR5	Large-scale simulations
Storage	1TB NVMe SSD	Fast asset loading

Installation

Isaac Sim Installation

#!/bin/bash
# NVIDIA Isaac Sim Installation Script

# Prerequisites
sudo apt update
sudo apt install -y wget gnupg2 software-properties-common

# Add NVIDIA repository
wget -qO - https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit.gpg
wget -q -O - https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
  sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit.gpg] https://#g' | \
  sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

# Install NVIDIA Container Toolkit
sudo apt update
sudo apt install -y nvidia-container-toolkit

# Configure Docker for GPU support
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

# Install Isaac Sim (via Omniverse Launcher or pip)
pip install isaacsim==4.0.0

# Verify installation
python3 -c "from omni.isaac.kit import SimulationApp; print('Isaac Sim installed successfully')"

Docker Installation (Recommended)

# Pull Isaac Sim container
docker pull nvcr.io/nvidia/isaac-sim:4.0.0

# Run Isaac Sim container
docker run --gpus all -it --rm \
  -e "ACCEPT_EULA=Y" \
  -e "PRIVACY_CONSENT=Y" \
  -v ~/nvidia-omniverse/cache/ov:/root/.cache/ov:rw \
  -v ~/nvidia-omniverse/data:/root/.local/share/ov/data:rw \
  -v ~/nvidia-omniverse/logs:/root/.nvidia-omniverse/logs:rw \
  -v ~/nvidia-omniverse/carp/cache:/root/.cache/unrealengine:rw \
  -v ~/isaac-sim:/root/isaac-sim:rw \
  --net=host \
  nvcr.io/nvidia/isaac-sim:4.0.0

Isaac Sim Architecture

Platform Components

┌─────────────────────────────────────────────────────────────────┐
│                      Isaac Sim Platform                          │
├─────────────────────────────────────────────────────────────────┤
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────────┐       │
│  │   Omniverse  │  │   Physics    │  │     Rendering    │       │
│  │   Connector  │  │   (PhysX 5)  │  │     (RTX)        │       │
│  └──────────────┘  └──────────────┘  └──────────────────┘       │
│                                                                  │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────────┐       │
│  │   ROS 2      │  │   Synthetic  │  │   Reinforcement  │       │
│  │   Bridge     │  │   Data       │  │   Learning       │       │
│  └──────────────┘  └──────────────┘  └──────────────────┘       │
│                                                                  │
│              ┌──────────────────────────────┐                    │
│              │      Python API / Extensions │                    │
│              └──────────────────────────────┘                    │
└─────────────────────────────────────────────────────────────────┘

USD (Universal Scene Description)

USD is the foundation of Isaac Sim scenes:

# Basic USD scene creation
from omni.isaac.kit import SimulationApp

# Launch Isaac Sim
simulation_app = SimulationApp({"headless": False})

from omni.isaac.core import World
from omni.isaac.core.prims import XFormPrim
from pxr import Usd, UsdGeom, Gf

# Create new stage
world = World(stage_units_in_meters=1.0)

# Add ground plane
world.scene.add_default_ground_plane()

# Add a cube
cube = world.scene.add(
    XFormPrim(
        prim_path="/World/Cube",
        name="cube",
        position=[0, 0, 1],
        scale=[0.5, 0.5, 0.5]
    )
)

# Run simulation
for i in range(1000):
    world.step(render=True)

simulation_app.close()

Basic Simulation Example

Python Script for Isaac Sim

#!/usr/bin/env python3
"""
Isaac Sim Basic Simulation
Creates a simple scene with a robot and runs physics simulation
"""

from omni.isaac.kit import SimulationApp

# Launch Isaac Sim before any other imports
simulation_app = SimulationApp({"headless": False})

from omni.isaac.core import World
from omni.isaac.core.prims import RigidPrim, XFormPrim
from omni.isaac.core.robots import Robot
import numpy as np

def main():
    # Create world
    world = World(
        stage_units_in_meters=1.0,
        physics_dt=1.0/60.0,
        rendering_dt=1.0/60.0
    )
    
    # Add ground plane
    world.scene.add_default_ground_plane()
    
    # Add light
    world.scene.add(
        XFormPrim(
            prim_path="/World/Light",
            name="light"
        )
    )
    
    # Add robot (URDF import)
    robot = world.scene.add(
        Robot(
            prim_path="/World/Humanoid",
            name="humanoid_robot",
            translation=np.array([0, 0, 0]),
            orientation=np.array([1.0, 0.0, 0.0, 0.0]),
            scale=np.array([1.0, 1.0, 1.0])
        )
    )
    
    # Initialize world
    world.reset()
    
    # Simulation loop
    for i in range(1000):
        # Get robot state
        position, orientation = robot.get_world_pose()
        
        # Apply control
        if i % 100 == 0:
            print(f"Step {i}: Position={position}, Orientation={orientation}")
        
        # Step simulation
        world.step(render=True)
    
    print("Simulation completed")
    simulation_app.close()

if __name__ == "__main__":
    main()

ROS 2 Integration

Isaac Sim ROS 2 Bridge

# ROS 2 bridge configuration
from omni.isaac.ros2_bridge import ROS2Bridge

# Create bridge
bridge = ROS2Bridge()

# Publish camera images
bridge.create_publisher(
    topic_name="/camera/color/image_raw",
    message_type="sensor_msgs/msg/Image"
)

# Subscribe to joint commands
bridge.create_subscription(
    topic_name="/joint_commands",
    message_type="sensor_msgs/msg/JointState",
    callback=joint_command_callback
)

Launch Configuration

# launch/isaac_sim.launch.py

from launch import LaunchDescription
from launch.actions import ExecuteProcess, DeclareLaunchArgument
from launch.substitutions import LaunchConfiguration

def generate_launch_description():
    isaac_sim_path = LaunchConfiguration('isaac_sim_path')
    usd_path = LaunchConfiguration('usd_path')
    
    # Isaac Sim execution
    isaac_sim = ExecuteProcess(
        cmd=[
            isaac_sim_path,
            '--/isaac/sim/app/file', usd_path,
            '--/isaac/sim/app/window/width', '1920',
            '--/isaac/sim/app/window/height', '1080'
        ],
        output='screen'
    )
    
    return LaunchDescription([
        DeclareLaunchArgument('isaac_sim_path', default_value='/path/to/isaac-sim.sh'),
        DeclareLaunchArgument('usd_path', default_value='/path/to/scene.usd'),
        isaac_sim,
    ])

Resources

Documentation

• Isaac Sim Documentation
• Omniverse USD Guide
• Isaac Sim ROS 2 Bridge

Tutorials

• Isaac Sim Getting Started
• Importing URDF to Isaac Sim

Exercises

1. Installation: Install Isaac Sim and verify with a basic scene
2. USD Creation: Create a simple USD scene with primitives
3. URDF Import: Import a URDF robot model into Isaac Sim
4. ROS 2 Bridge: Set up ROS 2 communication with Isaac Sim

Next Steps

Proceed to Week 9 - Isaac ROS Integration for advanced ROS 2 workflows.