search

haosulab / ManiSkill

SAPIEN Manipulation Skill Framework, a GPU parallelized robotics simulator and benchmark
https://maniskill.ai/
Apache License 2.0
722 stars 128 forks source link

Using TableTopFreeDraw-v1 env in ManiSkill2 #560

Open Adaickalavan opened 6 days ago

Adaickalavan commented 6 days ago

This question follows from issue #551, where a TableTopFreeDraw-v1 environment and a robot panda_stick was created for use in ManiSkill3.

If our current code base is confined to ManiSkill2, how can we use the TableTopFreeDraw-v1 and what modifications would be needed?

Any simple instructions on how to use TableTopFreeDraw-v1 environment in ManiSkill2 is much appreciated.

StoneT2000 commented 6 days ago

You can try adapting the code over but it won't be super trivial unless you know how to build tasks in ManiSkill2 and 3 (the differences being GPU sim stuff). You cannot run the MS3 environment code in the MS2 version.

I strongly recommend moving to ManiSkill 3 as soon as possible as it is difficult to maintain the previous version. Moreover MS3 should have the same gym interface as MS2 so I hope there is not too much friction with upgrading maniskill to the newest version. We have visual ml baselines you can try running directly on MS3 now.

Otherwise the general trick is to spawn e.g. 1000 little dots (they are cylinders with fixed radius and a small height) inside a floor/table so it can't be seen. During runtime at each environment step before fetching observations then teleport the dot to the surface so it can be seen and teleport it right under where the panda's TCP position is.