search

isaac-sim / IsaacGymEnvs

Isaac Gym Reinforcement Learning Environments
Other
1.89k stars 409 forks source link

Is it possible to use Isaac Gym without simulating physics? #21

Closed hmomin closed 2 years ago

hmomin commented 2 years ago

Hi all, thank you for the awesome work you've done in creating Isaac Gym!

I'm wondering if it's possible to use Isaac Gym without making use of any physics simulation.

More specifically, it seems that dof_state_tensor and actor_root_state_tensor all require physical actors to be simulated with gravity, ground planes, etc... but what if we'd like to train with Isaac Gym on a non-physically-based RL task? For example, the CarRacing environment from OpenAI or some other task in healthcare, transportation, finance, etc.?

I would think it's possible to do this with torch.jit functions by:

  1. initializing random state tensors for each environment,
  2. applying actions and stepping the environment in the pre_physics_step(),
  3. computing rewards and observations in the post_physics_step(),
  4. repeating all this as desired.

However, I've gone through a lot of the documentation and haven't found any way to initialize states without specifying some sort of physical actor. Specifying actors wouldn't make sense if we were trying to train on CarRacing, for example.

The fully accelerated RL paradigm in Isaac Gym is extremely powerful and I feel it would be huuugely beneficial to the RL community to be able to train any task fully on the GPU, even if non-physical.

Any thoughts or advice?

Thank you, Momin

Yangxiaojun1230 commented 2 years ago

I am also looking at transfer RL task from CPU clusters to GPU that doesn't require physical actor.

gavrielstate commented 2 years ago

The key thing that Isaac Gym keeps on the GPU is the physics simulation. If you are not running physics simulation at all, and your environment stepping is something other than a physics simulator, you would need to figure out some way to do your environment stepping on the GPU. Unfortunately, torch.jit often doesn't produce very fast GPU code. I don't think this is going to get you where you want to go.

That said, I think that NVIDIA WARP may be a solution for these kinds of needs. You could probably adjust other existing RL environments to use WARP. See more tomorrow: https://www.nvidia.com/gtc/session-catalog/?tab.scheduledorondemand=1583520458947001NJiE&search=warp#/session/16384065793850013gEx

We'll be exploring WARP with Isaac Gym and physical simulation as well, but non-physics RL is not our main focus.