How Tos¶
Create Simulation Scene¶
You want to design and set up a simulation scene that replicates specific environments. There are several ways to achieve this on the EXPLORE Platform with Isaac Sim.
Creating a new scene¶
To create a new, empty scene, first launch Isaac Sim: /isaac-sim/isaac-sim.sh $STREAMING_ARGS
Then, go to File > New. Further information is accessible in the official documentation.
Simulating Human Actors¶
A workstation for manual assembly involves a human worker, tools, materials, and work instructions. Humans are easily simulatable in Isaac Sim.
Replicator¶
The Replicator extension provides scriptable human actors for the simulation with IsaacSim.Replicator.Agent (IRA). These actors can execute simple commands, like walking to a point (x,y,z) or executing animations. The guide on how to do this is accessible in the official documentation.
Generate USD Model from CAD Files¶
For importing your own models in Isaac Sim you need a way/an instruction to convert existing CAD files into USD (Universal Scene Description) models to ensure a robot’s or an asset's appearance and material properties are accurately represented in simulation environments.
Current solution¶
Isaac Sim offers direct support for importing most ordinary CAD file formats. You can import them via File > Import. For more complex models like robots, at the current point in time, we do not offer a direct pipeline. Online guides suggest converting the CAD to URDF first, and then using the URDF importer of Isaac Sim.
Import a robot from URDF into Isaac Sim¶
To import a custom robot to Isaac Sim you usually import an URDF. This allows an easy on-boarding of robots that have been defined in the ROS universe.
URDF Importer¶
Isaac Sim has a built-in URDF importer. You can either access it through the File > Import menu or by selecting a URDF file in the file explorer and choosing the import option. The importer has several options to configure the created USD file. For more information, refer to the official documentation.
Interact with custom ROS data¶
The built-in selection of ROS/ROS2 messages and services is limited and you might need to use additional message types, publishers, subscribers, service calls and more. There are multiple ways to include these in your simulation.
ROS Bridge extensions¶
It is possible to define custom Action Graph nodes to use with the other ROS Bridge nodes. The official documentation contains guides for creating new C++ ROS extensions and Python ROS extensions.
Direct scripting¶
It is also possible to access ROS through scripting, which is especially relevant in the Extension workflow and in the Standalone workflow (see different Isaac Sim workflows). The python custom message scripting tutorial is a good resource.
Generic publishers and subscribers¶
You can also attempt to build your messages directly in the Action Graph from the built-in data types and publish them through a generic publisher or subscribe them with a generic subscriber. A guide for this is accessible here.
Enable Domain Randomization in Simulation¶
Isaac Sim and Isaac Lab allow to execute multiple simulation runs while varying key parameters such as environmental conditions, physics properties, sensor noise, and object interactions.
Isaac Sim¶
There are several resources for domain randomization in Isaac Sim. One good starting point is to look into the documentation of the Replicator Framework.
Isaac Lab¶
Isaac Lab, the parallel learning framework based on Isaac Sim, provides a different set of tools for domain randomization. The Getting Started with Isaac Lab tutorial series has a helpful chapter on Sim-to-Real transfer that contains a lot of guides this topic.
Interact with ROS Middleware¶
Interaction between Isaac Sim and ROS/ROS2 is built-in and several guides explain in detail how to develop simulations with the ROS ecosystem.
The Isaac Sim - ROS communication is provided by the ROS Bridge/ROS2 Bridge extension. If not enabled by default, it is accessible through the Window > Extensions menu. After that, you can use ROS/ROS2 nodes in action graphs. Examples are provided in the official documentation. The environment variables are set by default on an EXPLORE instance. For certain standard behaviors, a generator can be used to create the ROS2 Action Graphs automatically. For that, go to Tools > Robotics > ROS 2 OmniGraphs.
Internally, the bridge has the following execution sequence: 1. Check ROS installation and environment, initialize rclpy from system environment or internal libraries if not possible. OmniGraph nodes are made available through loading the extension. 2. Every OmniGraph Node computes its data and publishes/subscribes every tick.
Natively included topics
Isaac Sim currently supports publishing the following ROS1 topic types by default from ActionGraphs:
- AckermannDrive
- Bbox2D / Bbox3D
- Camera Info
- Clock
- Image
- Imu
- Joint State
- Laser Scan
- Odometry
- Point Cloud
- Raw Transformation Tree
- Transform Tree
- Semantic Labels
Isaac Sim currently supports subscribing the following ROS1 topic types by default from ActionGraphs:
- AckermannDrive
- Clock
- Joint State
- Twist
Isaac Sim currently supports providing the ROS1 Teleport service by default from ActionGraphs. If any other Topics or Services need to be used, please refer to Interact with custom ROS data
Further Information¶
Extend Isaac Sim via the extension workflow¶
One of the three main workflows for Isaac Sim is the Extension workflow. This workflow extends the simulation's functionalities by including a hot-reloadable python extension in the Omniverse framework. For mor information, use the links in the attachments.
Further Information¶
Extend Isaac Sim via the standalone workflow¶
Standalone applications utilizing Isaac Sim are written in Python. This workflow provides full control over the simulation loop. Furthermore, the simulation can be started headlessly without the user interface. This is one of the three main workflows for Isaac Sim. For more information, refer to the links in the attachments below.
Further Information¶
Create a gripper that can freely float in 6D space¶
Load the gripper USD file in Isaac Sim GUI. Create a parent XForm that includes all joint and links of the gripper. Ensure that the gripper has prismatic and revolute joints to cover all six DoF so that it can freely float and add an articulation root to the parent XForm.
Add an Actuator to Robot Model in USD Format¶
Ensure the robot is an articulation. Select the desired joint prim to add drive and configure properties. Use Isaac Sim APIs to set joint targets per step or edit joint drive properties while the simulation is running.
For additional information refer to: Articulation Controller
Integrate collision meshes to my model¶
To add collision to a mesh you select the mesh, add a collider and then choose the desired approximation. To create a simple collision mesh, first create a geometry like a cube. Change position and size to desired values, then add a collider. Optionally set visiblity to false.
For additional information refer to: Physics Collision Extention Robot Setup Tutorials
Add a Sensor to Robot Model in USD Format¶
To add a sensor to a Model first create the desired sensor. Then place the sensor at its appropriate position and add the sensor to its place in the stage tab hierarchy. Finally edit properties if so desired.
For additional information refer to: Adding Sensors
