Notes on using Issac Sim Python API
Issac Sim provides a set of APIs that are relevant for robotics applications. They abstract the complexities of dealing with Pixar's USD Python API.
BaseSample – A boilerplate extension application that sets up the basics for a robotic extension application.
An extension application is a UI elements provider in Issac Sim parlance.
- Loading the world and it's assets.
- Clearing the world when a new stage is added.
- Reset the world to default state.
- Handle hot reloading.
BaseSample has a World instance. World allows to interact with the simulator in an easy and modular way. It handles time related events(including physics step), resetting scene and adding tasks etc. World is a singleton and the reference to it can be obtained anywhere as
World.instance() (after the BaseSample has been initialized in the constructor).
A World contains an instance of a Scene. Scene manages simulation assets in the USD Stage. Scene provides an API to manipulate different USD assets in the stage.
When creating a BaseSample one can use
setup_scene to initialize scene for the first time use.
NOTE: setup_scene is not called during hot reloading but only during loading the world from an EMPTY stage.
Objects can be added to the scene using
setup_post_load is called when load button is pressed, after
setup_scene and after one physics time step.
When pressing the play button for simulation we can get a registered callback triggered using World's
add_physics_callback which takes the callback function as the parameter. This function is called just before the physics time step is executed.
Instead of an extension application one can also create a standalone application where the application is stated from python and one can control when to step physics and rendering.
After creating a
SimulationApp one can add the world, scene and objects in the scene as usual. Then using
world.step(render=True) one can execute a single physics time step.
Adding a robot to the stage
Issas Sim provides a number of helper classes which make it easier to deal with USD assets. After adding the robot as USD to the stage using
add_reference_to_stage one can wrap the USD asset under Robot and add Robot to the scene.
Robot is an Articulation which according to the comments in the source code provides high level functions to deal with an articulation prim and its attributes/ properties.”
Every Articulation has an ArticulationController which according to it's comments is a PD Controller of all degrees of freedom of an articulation, can apply position targets, velocity targets and efforts.
During a physics step(which we have access to using callbacks as state above) we can apply an ArticulationAction to the ArticulationController to specify joint positions, efforts or velocities.
Instead of using the Robot class one can also use a more specialized version called
WheeledRobot which derives from the Robot class. This class then provides a more specialized
apply_wheel_action instead of more general
apply_action of the Robot class.
Adding a custom controller
Controllers inherit from BaseController and must implement
forward method which receive a whatever command and returns
Once a custom controller has been created, this can be assigned as the controller of the robot. The robot can
apply_action on the
ArticulationAction as returned by the forward method.
Before implementing custom controllers it is always wise to check ready made controllers such as
Manipulator robots have their own domain specific controllers such as
PickAndPlaceController. The forward method of this controller for example takes a picking position, a placing position and the current joint position and calculates a step forward and returns an
ArticulationAction which the robot can apply.
For creating complex scenes one can use
Tasks which provides a way to modularize scene creation. Custom tasks derive from
BaseTask and implement
get_observationswhich Returns current observations from the objects needed for the behavioral layer. The task can implement
post_reset etc as usual.
Task can be added to the World which will then call task's
setup_scene when initializing the scene.
Tasks help encapsulate the logic and hence make it easier to write larger more complex scenes. Just as in adding custom controllers, Issac Sim provides common tasks which should be used if relevant. For example PickPlace task for Franka.
Tasks can have other tasks in them. In this case the subtask's
setup_scene can be called by the parent task.
Personally, I feel Task is not aptly named. It should be called SubScene perhaps.
Multi robot scene
Just a matter of maintaining a state machine and orchestrate the tasks as needed in the physics time step. Each robot has it's own controller and requires an ArticulationAction as needed to make the whole scene and handoff work as needed.