Topic 4 — Target Platforms & Sim-to-Real Workflow
This topic expands the Target Platform research section into a concrete workflow for how you will use:
- A powerful Digital Twin Workstation (Ubuntu 22.04 + RTX GPU), and
- A resource‑constrained Physical AI Edge Kit (Jetson Orin Nano),
to develop, test, and deploy humanoid behaviors.
It connects directly to the course’s Simulation‑First and Sim‑to‑Real principles.
1. Digital Twin Workstation (Ubuntu 22.04 + RTX GPU)
Your primary development machine is an Ubuntu 22.04 LTS workstation with an NVIDIA RTX GPU.
On this machine you will:
- Run ROS 2 (Humble) as the main middleware.
- Run Gazebo and NVIDIA Isaac Sim for digital twin simulation.
- Execute heavy perception and ML workloads (e.g., object detection, SLAM, training).
- Develop and test Whisper + LLM pipelines for natural language commands.
Why Ubuntu 22.04?
- Stable and officially supported by ROS 2 Humble and NVIDIA toolchains.
- Common in academic and industrial robotics labs.
- Matches the environment assumed by most reference docs and tutorials.
2. Physical AI Edge Kit (Jetson Orin Nano)
The Jetson Orin Nano represents your on‑robot compute:
- Runs ROS 2 nodes with tighter compute and power budgets.
- Connects to physical sensors:
- Intel RealSense D435i/D455 (RGB‑D + IMU).
- Microphones, optional IMUs, and other peripherals.
- Hosts only the runtime pieces needed for deployment:
- Navigation stacks.
- Perception models trimmed for edge execution.
- Simple VLA bridges where feasible.
Why Jetson Orin?
- Strong GPU for real‑time perception on the edge.
- Widely used in robotics education and research.
- Forces you to think about model size, latency, and power, not just accuracy.
3. Sim-First, Deploy-Second: The Workflow
The recommended development loop is:
-
Prototype in Simulation (Workstation)
- Design URDF models, launch files, and initial controllers in ROS 2.
- Use Gazebo or Isaac Sim to validate physics, perception, and navigation.
-
Tighten the Loop with Isaac Sim
- Move to higher‑fidelity Isaac Sim scenarios for advanced perception and VSLAM.
- Generate synthetic data for training and evaluation.
-
Performance Profiling
- Measure CPU/GPU usage, latency, and frame rates on the workstation.
- Identify which components must be optimized or approximated for the edge.
-
Sim-to-Real Transfer
- Package the required ROS 2 nodes and models for Jetson Orin.
- Deploy via containers, SSH, or flash images (as per lab instructions).
-
On-Robot Validation
- Run the same ROS 2 APIs used in simulation, now backed by real sensors.
- Log failures, regressions, and timing issues, then iterate.
This loop ensures that most bugs are caught in simulation, and that deployment to physical hardware is a controlled, incremental step.
4. Keeping Interfaces Consistent Across Workstation & Edge
To make sim‑to‑real transfer manageable:
- Use the same ROS 2 package structure on both machines.
- Keep topic names, service definitions, and actions identical between sim and real deployments.
- Factor hardware dependencies behind clear abstraction layers:
- e.g.,
/camera/depthshould work whether it is a Gazebo plugin or a real RealSense camera.
- e.g.,
If your code treats “where the data comes from” as an implementation detail, you can:
- Swap simulators (Gazebo vs Isaac) with minimal changes.
- Switch from simulated to real sensors without rewriting core logic.
5. How Assessment Maps to Platforms
Throughout the course:
- Early labs focus almost entirely on the workstation + simulation:
- ROS 2 basics, Gazebo setups, simple Isaac Sim scenes.
- Mid‑course labs begin to explore edge deployment:
- Running perception and navigation nodes on Jetson with recorded data.
- Capstone combines both:
- Isaac Sim + ROS 2 on the workstation.
- Selected behaviors deployed to Jetson for physical‑world experiments where available.
Understanding the roles of both platforms, and how to move code between them, is essential for succeeding in the capstone and in real‑world robotics projects.