Using TableTopFreeDraw-v1 env in ManiSkill2

[Adaickalavan]

This question follows from issue #551, where a TableTopFreeDraw-v1 environment and a robot panda_stick was created for use in ManiSkill3.

If our current code base is confined to ManiSkill2, how can we use the TableTopFreeDraw-v1 and what modifications would be needed?

Any simple instructions on how to use TableTopFreeDraw-v1 environment in ManiSkill2 is much appreciated.

[StoneT2000]

You can try adapting the code over but it won't be super trivial unless you know how to build tasks in ManiSkill2 and 3 (the differences being GPU sim stuff). You cannot run the MS3 environment code in the MS2 version.

I strongly recommend moving to ManiSkill 3 as soon as possible as it is difficult to maintain the previous version. Moreover MS3 should have the same gym interface as MS2 so I hope there is not too much friction with upgrading maniskill to the newest version. We have visual ml baselines you can try running directly on MS3 now.

Otherwise the general trick is to spawn e.g. 1000 little dots (they are cylinders with fixed radius and a small height) inside a floor/table so it can't be seen. During runtime at each environment step before fetching observations then teleport the dot to the surface so it can be seen and teleport it right under where the panda's TCP position is.

[StoneT2000]

@Adaickalavan has your issue been resolved? Let me know if there are any other problems with the setup.

[Adaickalavan]

I realised that the controllers are well abstracted and can be easily ported between ManiSkill2 and ManiSkill3, requiring only slight code changes involving the imports, attributes, and methods.

Instead of porting the TableTopFreeDraw-v1 env from ManiSkill3 to ManiSkill2, I see that it is easier to port the controller code from ManiSkill2 to ManiSkill3.

Therefore, this issue can be closed.