search

pytorch / rl

A modular, primitive-first, python-first PyTorch library for Reinforcement Learning.
https://pytorch.org/rl
MIT License
2k stars 268 forks source link

[Feature Request] Support for Gymnasium #613

Open pseudo-rnd-thoughts opened 1 year ago

pseudo-rnd-thoughts commented 1 year ago

Motivation

Gymnasium is a fork of the OpenAI Gym project by that same people that maintained Gym for the last 18 months. Our understanding from OpenAI is that they are not planning on maintaining Gym currently therefore, we would recommend using Gymnasium.

For more information about this, see our announcement - https://farama.org/Announcing-The-Farama-Foundation

Solution

Currently, the Gymnasium and Gym are identical repos so should be compatible between the two projects.

try:
   import gymnasium as gym
except ImportError:
   import gym

However in the future, we are planning on adding a new feature improvements that this repo would be interested in

vmoens commented 1 year ago

Thanks for this @pseudo-rnd-thoughts I like the idea! We're as excited as can be about gymnasium and looking forward to adopting it. We're in the process of making the library compatible with gym 0.13 and on -- quite a herculean task! I would be glad to have your thoughts on that.

 Some context:

Currently, for instance, we do not support the new step method (which I believe returns 2 Booleans instead of one done state). This is something we should work on, I'm planning to do that once we have our versioning utils in place.

Many of our users stick to a specific gym version (e.g. habitat or robotic people) and they're happy with it. As a SOTA library we obviously want to support latest changes too!

Versioning

I see that gymnasium starts at 0.26.3, meaning that it's one step ahead of gym. So I guess we'd be good with the code snippet you provided above (since we'll be calling if parse(gym.__version__) < smth frequently). Therefore I guess that our end it's mostly going to be just a matter of replacing gym import in the gym.py file and changing the CI configuration to install gymnasium instead.

I guess we can take care of that. I'll keep you posted on the progress.

pseudo-rnd-thoughts commented 1 year ago

Thanks for the quick response. That works for us

Do you mind me asking why 0.13 is the minimum? It is a 3 year old version with a number of critical differences in the core API between 0.13 and the latest version. In addition to the bug fixes and features that it is missing, plus it only supports python 3.7.

Edit: A migration guide to v26 and we have a legacy compatibility wrapper from v21 to v26 wrappers in gym / gymnasium v26 https://gymnasium.farama.org/content/migration-guide/

jkterry1 commented 1 year ago

For what it's worth, if you want to keep old API compatibility and a more supported older version of Gym, we'd recommend using 0.21. Upgrading from 0.13 to 0.21 shouldn't even be especially difficult

vmoens commented 1 year ago

Do you mind me asking why 0.13 is the minimum? It is a 3 year old version with a number of critical differences in the core API between 0.13 and the latest version. In addition to the bug fixes and features that it is missing, plus it only supports python 3.7.

Mainly hardware requirements. Some libs for robot control have been written with gym 0.13 and upgrading to a new gym version is going to be hard for these users -- once it holds you don't touch it.