search

facebookresearch / habitat-lab

A modular high-level library to train embodied AI agents across a variety of tasks and environments.
https://aihabitat.org/
MIT License
1.9k stars 476 forks source link

Feature request: migrate from gym to gymnasium #1213

Open elliottower opened 1 year ago

elliottower commented 1 year ago

🚀 Feature

Migrate from gym (no longer maintained) to gymnasium.

Motivation

Gymnasium a maintained fork of openai gym and is designed as a drop-in replacement (import gym -> import gymnasium as gym). Beyond just bugfixes, many RL training libraries have also switched (rllib, tianshou, CleanRL), or are planning to switch ((stable-baselines3](https://github.com/DLR-RM/stable-baselines3/blob/e5deeed16efb57c34ccdcb14692439154d970527/docs/guide/install.rst#bleeding-edge-version)). It would be great if users could train agents on Habitat using the latest models and features from these libraries (e.g., scalable distributed training/model serving using Ray/RLlib).

Pitch

For information about upgrading and compatibility, see migration guide and gym compatibility. The main difference is the API has switched to returning truncated and terminated, rather than done, in order to give more information and mitigate edge case issues (for example, many popular tutorials/implementations of Q learning using gym were actually incorrect because of done, there will be an upcoming blog post explaining more details about this on the Farama site (https://farama.org/blog).

Alternatives

An alternative is to keep using gym internally and write a wrapper to be compatible with these other libraries, but migration to gymnasium should not be too difficult and it would be cleaner to avoid the extra dependency. A compatibility wrapper could also be made using Shimmy, but this is mainly for non-gym projects such as DeepMind Control, DeepMind Lab, etc.

Additional context

We would love to add this repository to our list of third party environments , as it is a major research tool, but are only including repositories which are compatible with gymnasium.

Being able to use habitat environments with gymnasium would also enable a lot of research. It could potentially be integrated into gymnasium-robotics as well, which would be helpful for researchers.

aclegg3 commented 1 year ago

We are interested in learning more about the steps involved in this migration and the benefits of doing so. We are also happy to accept contributions and to review PRs adding additional support such as this.

elliottower commented 1 year ago

We are interested in learning more about the steps involved in this migration and the benefits of doing so. We are also happy to accept contributions and to review PRs adding additional support such as this.

Great to hear, I could look into it at some point in the next few weeks and see how difficult it looks. It seems you’re using gym v22 which uses a done signal rather than separate termination and truncation signals which are used in v26 and in gymnasium. As for benefits of migration all major training libraries have transitioned to Gymnasium (RLlib, Stable-Baselines3, CleanRL, Tianshou, Google’s Dopamine is planning on it) so it would be desirable for users and researchers to work with your environments. Being unmaintained there are also installation issues on certain systems (M1 and other ARM professors for example, also I know Gym V21 relies on dependencies which are also unmaintained which leads to problems), for Gymnasium we prioritize compatibility with current training libraries and other libraries like numpy/torch, python 3.11, etc.

We are also finalizing a blog post explaining more about termination and truncation, which explains why the previous way of doing things led to incorrect implementations of algorithms and other unncaught bugs.