[Question] Importing Partnet-mobility dataset

[dHonerkamp]

Hi, thanks for the great work of putting the paper, library and benchmark together.

In the paper you mention that Orbit supports both YCB and Partnet-Mobility. I found the the example to use YCB (play_ycb_objects.py) but didn't find any details on Partnet-Mobility. I also couldn't find them looking through the assets in the Isaac nucleus.

Do you have any details on how to import Partnet-Mobility into Isaac / Orbit? Thanks a lot.

[Mayankm96]

Hi!

Uploading the entire partnet-mobility (SAPIEN) dataset on Nucleus was proving to be rather expensive, so we didn't end up uploading the assets over there. We are working now on making an extension in Orbit that loads the dataset and also annotates part semantics (which might be useful for perception).

The expected timeline for this will be around the end of May. But if you need a quicker solution, I suggest downloading the dataset from where2act paper that has the SAPIEN dataset with convex decomposition using VHACD.

https://github.com/daerduoCarey/where2act

Form from their website to get access to their dataset: https://docs.google.com/forms/d/e/1FAIpQLSegEvIM22Ta44MrKM5d-guRE4aDR5K77ZQoInLWEyib-aeCFw/viewform

After that, you can import these assets using the URDF importer in Isaac Sim:

https://docs.omniverse.nvidia.com/app_isaacsim/app_isaacsim/ext_omni_isaac_urdf.html

Maybe the discussion in #48 might be useful here too.

[dHonerkamp]

Hi Mayank,

Thanks a lot, that's very helpful. Looking forward to the next releases.

[dHonerkamp]

Hi Mayank,

This worked very well and I am able to import all the objects. But I am running into an issue in which "phantom collisions" between the robot and the object happen. In particular, as soon as I import a second articulation (in addition to the robot) and then call self._robots.set_joint_positions(), in the very next physics step there happens a collision which shakes or tops over the articulated object - even if the object is 10 meter away.

I have raised this with nvidia directly (last post in https://forums.developer.nvidia.com/t/robot-reset-failing-if-in-contact-or-simultaneously-resetting-object-since-v2022-2-0/248452/8). But I was wondering if you have experienced anything similar when you were running the examples in the Orbit paper?

[Mayankm96]

Hi!

We saw this too in one of our cabinet environments as well (thus the reason why we haven't open-sourced it yet). From what I have seen, there has been a regression in physics ArticulationView from Isaac Sim 2022.2.0 to Isaac Sim 2022.2.1 and I think that's the root cause of these failures.

Can you try running with 2022.2.0 and see what happens? Also, if you run with CPU physics (i.e. --cpu flag), I think you'd see the "correct" behavior. If not, then the issue is arising from something else here.

Also based on the video, you should probably import the cabinet with a fixed joint. Then it won't topple.

[dHonerkamp]

That's very good to know, thanks.

Indeed, this does not happen if running on the CPU. I think I'm seeing some improvements after converting the usd files (including our robot) to instanceable, but there is still some weird behavior. I also added a fixed joint, but still ended up with the cabinet doors opening / closing at random points

I will try with 2022.2.0 as well. You wouldn't happen to know if this regression will be addressed in the next Isaac release?

[Mayankm96]

Yes, from what I know, it should get fixed in the coming release. I'll check with their upcoming release as soon as its available.