search

isaac-sim / IsaacLab

Unified framework for robot learning built on NVIDIA Isaac Sim
https://isaac-sim.github.io/IsaacLab
Other
2.16k stars 885 forks source link

[Question] Does it support external 3d scenes? #167

Closed user432 closed 8 months ago

user432 commented 11 months ago

Hi, thanks for actively maintaining this huge code base. I would like to know if Orbit supports loading external 3d scenes (like this) to train mobile manipulation agents?

Mayankm96 commented 11 months ago

Hi, yes, it should be possible to import these kind of scenes into Orbit. Internally, we have been experimenting with Matterport-3D dataset and have managed to load them into the simulation. The most important part is to generate the USD for those scenes.

Omniverse supports different asset importers. If you can provide information on what format the dataset is in, I can provide you pointers to the right extensions to investigate.

user432 commented 11 months ago

Hi Mayank, thanks for your prompt reply. Originally, they're in glb format and I have successfully converted a few of them into USD files using Omniverse's USD composer tool.

Mayankm96 commented 11 months ago

In that case, importing those assets into Orbit should be straightforward. You can use the UsdFileCfg class for this:

https://github.com/NVIDIA-Omniverse/Orbit/blob/devel/source/extensions/omni.isaac.orbit/omni/isaac/orbit/sim/spawners/from_files/from_files_cfg.py#L51

Mayankm96 commented 8 months ago

Closing this issue due to inactivity. Please re-open if the issue still persists.

SourabhPrasad commented 1 month ago

Hi @Mayankm96, I am trying to import the Matterport3D dataset to Isaac Sim 4.1.0. Could you please advise on how you were able to load the dataset?

Mayankm96 commented 1 month ago

Hi @SourabhPrasad , yes it does :)

You can look at one of our projects that imports scenes from various places (Matterport, Unreal Engine): https://github.com/leggedrobotics/viplanner

I hope that helps!

SourabhPrasad commented 1 month ago

Thanks for the reply, @Mayankm96! I have converted the .obj file to .usd, but it doesn't contain the lighting information. In your implementation, were the lights added manually after conversion to .usd?