search

openai / gym

A toolkit for developing and comparing reinforcement learning algorithms.
https://www.gymlibrary.dev
Other
34.49k stars 8.59k forks source link

[Bug Report] Auto-reset on Env instantiation is backward-incompatible #2862

Closed yannbouteiller closed 2 years ago

yannbouteiller commented 2 years ago

Describe the bug The Env class now calls reset by default on instantitation, which breaks a useful behavior.

Additional context

Hello. Recently I had to deal with many breaking changes with Gym introducing bugs in my libraries. I could deal with most of them but I am not sure what to do about this one.

I maintain rtgym, a library helping people create Gym environments for robots in the real world. Many implemented RL algorithms instantiate a Gym environment for the sole purpose of retrieving the action and observation spaces. Needless to say that in the real world, you cannot have a robot operating just for that, especially when your training algorithm is on a HPC cluster. Thus, the workaround is to call an initializer once in reset() to instantiate everything that cannot be instantiated on a HPC, relying on the fact that reset() is not called on Env instantiation.

This has been broken by the new default behavior of the Env object, which now autoresets on instantiation.

Checklist

yannbouteiller commented 2 years ago

Closing as there is a workaround described in the release notes (disable the checker on Gym.make)

pseudo-rnd-thoughts commented 2 years ago

Thanks for the issue and fixing it yourself We have realised that the environment check is causing more issues than will fix so in the next release, hopefully, v24.1 then environment check will be modified to not cause this issue. See https://github.com/openai/gym/pull/2864 for the updated version