nicklashansen / tdmpc2

Code for "TD-MPC2: Scalable, Robust World Models for Continuous Control"
https://www.tdmpc2.com
MIT License
343 stars 71 forks source link

Compatibility with Latest Version of ManiSkill2 Based on Gymnasium #12

Closed TongZhangTHU closed 8 months ago

TongZhangTHU commented 8 months ago

Hi,

Thanks for your wonderful work!

I'm currently looking to use the latest version of ManiSkill2, which is now based on Gymnasium. Could you advise on the simplest method to achieve compatibility with this version? Specifically, I'm wondering if a wrapper or some specific parameters could be used for this purpose. (My focus is solely on adapting to ManiSkill2; compatibility with other environments is not required at this time.)

nicklashansen commented 8 months ago

Thanks for reaching out! I'll be happy to help you migrate to the latest version of ManiSkill2.

I have not worked with the newer versions of ManiSkill2 / Gymnasium before, but I believe that the main difference is the use of (terminated, truncated) in place of the now deprecated done signal. You should be able to use our code as-is if you simply set done = terminated or truncated and treat the problem as an infinite-horizon continous control problem (episodes are truncated at a fixed # episode steps; no early termination).

@Jiayuan-Gu and @StoneT2000 might be able to provide more details on the Gym -> Gymnasium migration of ManiSkill2.

StoneT2000 commented 8 months ago

There are two changes

One is the done change as mentioned by Nicklas

The other is that now the return value of env.reset is tuple containing observation and a empty dictionary

Simply just use the observation and ignore the empty dictionary.

Otherwise there aren't any significant breaking changes I can think of off the top of my head

nicklashansen commented 8 months ago

@StoneT2000 Thanks for your input!

TongZhangTHU commented 8 months ago

@nicklashansen @StoneT2000 Thank you so much for your reply and kind help!