search

microsoft / nni

An open source AutoML toolkit for automate machine learning lifecycle, including feature engineering, neural architecture search, model compression and hyper-parameter tuning.
https://nni.readthedocs.io
MIT License
14.07k stars 1.82k forks source link

Transition to Gymnasium / Future Compatibility with Tianshou #5308

Open Markus28 opened 1 year ago

Markus28 commented 1 year ago

The maintainers of OpenAI Gym recently forked Gym to Gymnasium, which is where all future maintenance will occur. More information can be found in the announcement blog post of the Farama Foundation.

Tianshou will, along with many other large RL frameworks, transition towards Gymnasium. We are currently working on a PR for this: https://github.com/thu-ml/tianshou/pull/789. In particular, Tianshou will stop supporting bare Gym environments, especially those that follow the old API.

NNI depends on Tianshou via PolicyBasedRL. The transition of Tianshou to Gymnasium would break this since ModelEvaluationEnv is an old-style Gym environment. This can easily be fixed by updating the environment to Gymnasium. Gymnasium is currently a drop-in replacement for OpenAI Gym, so this shouldn't be too hard. However, the environment should conform with the new API (already introduced in OpenAI Gym 0.26), which would require some simple changes, as explained here.

I'd be happy to help with any questions or PRs. @jkterry1 @Trinkle23897

matluster commented 1 year ago

There is coming new implementation of PolicyBasedRL in NNI v3.0. The large part of coding work is done and I just incorporate the new API of Gymnasium: https://github.com/microsoft/nni/commit/2efe0c6502d354d99181e05256823063458854ed

It will come into the master branch of NNI shortly. If tianshou has encountered any problems in CI, feel free to disable them temporarily. I'll address those problems after v3.0 is released.

Thanks for the notification.

dtamienER commented 5 months ago

What versions of Tianshou, Gym and shimmy could be used in a NNI v2.7 environment in order to work with RL?