Compatibility with new gym API + New wrapper for out-of-the box usage of RL libraries (e.g. stable_baselines3)

[saleml]

In recent updates of gym, the API has changed significantly, creating backward compatibility issues with minigrid and enviornments based on it.

• In the gym.make function, the argument disable_env_checker is by default set to False. This means that whenever an environment inheriting from MiniGridEnv is created using gym.make('env'), it is wrapped in PassiveEnvChecker which verifies at each time step that the observations are contained in the observation space of the environment. I thus wrote a simple gym.Space to handle textual observations, and changed the environment observation space accordingly.gym.spaces.Text allows for textual observations.

• When using gym_minigrid's wrappers, PassiveEnvChecker failed, because some wrappers did not only define a new observation space of their own, but changed the unwrapped environment's observation space. This is about the wrappers that change the image part of the observation space, by overwriting self.observation_space.spaces["image"], which is not the right way to change this part of the observation space, given that it modifies that of the unwrapped environment as well, which is undesirable. This should be fixed now.

• The current version of gym 0.24.1 has a bug though, related to PassiveEnvChecker, as described here and here; which will be fixed very shortly, as explained here. While it is obviously preferable to wait for the next version that introduces the right fixes, I slightly modified setup.py and .travis.yml to use the branch associated to this PR, that will certainly be the basis of gym's next version. As soon as gym is updated, I'll update this PR !

• Rendering is handled differently in gym 0.24.1. A warning is raised whenever self.render is called with mode argument. As you can see here: "The argument mode in render method is deprecated; use render_mode during environment initialization instead. See here for more information: https://www.gymlibrary.ml/content/api/". I made the corresponding change. Whenever rendering is expected, render_mode should thus be given to gym.make.

• I made a small change to the metadata keys of MiniGridEnv match the metadata keys of gym (as in here)

• Seeding: The seed function is deprecated, and seeding is handled by the reset method completely, rather than having to do both env.seed() and env.reset() at each time. I made the corresponding change. gym's rng won't handle randint soon (as shown in some warnings), and I changed the _rand_int, _rand_pos, and _rand_bool methods of MiniGridEnv accordingly.

• I added an argument to the method get_view_exts, given that one of the wrappers (ViewSizeWrapper) that changes the agent view size used to modify the unwrapped environment agent_view_size attribute, thus breaking the compatibility between the unwrapped environment's observation_space and the new generated observations. This required adding an argument to gen_obs_grid. I also changed ViewSizeWrapper accordingly. I am not sure this is the simplest way to do it though.

• In gym_minigrid/window.py, Matplotlib raises a deprecation warning when setting the window title. This should be fixed now.

• I added a new wrapper: DictObservationSpaceWrapper. Put simply, this wrapper changes the mission from text to array. For this, I manually (I found no cleaner way; can you think of one?) listed all words used in the MiniGrid language and encoded each textual mission using the corresponding word indices. The mission has a maximum length (number of words). The benefit of doing this is that a numerical observation space for the mission part of the whole observation space is necessary to use RL libraries out of the box, that transform the observations (here = image + direction + mission as array) to tensors. I also added some corresponding tests in run_tests.py.

- I moved EmptyEnvWithExtraObs, that was defined in run_tests.py, to empty.py in order to be able to register it correctly with gym (I couldn't figure out how to register it if it wa defined in run_tests.py). Registering environments seem to be essential with the changes to the gym API (in order to be able to use render_mode argument when calling gym.make for example)

• For consistency, the mission in FourRoomsEnv is now in lower-case letters, as in all other environments.

There should be no warning when running run_tests.py or manual_control.py.

[saleml]

I updated the setup and yaml files to use the new released version of gym (0.25). This new gym release also contains a Text spaces. There is no need for the StringGymSpace I had written anymore.

Note: With gym 0.25, the step function outputs a 5-tuple instead of a 4-tuple. gym raises a warning as minigrid still outputs a 4-tuple. I have not made the change to any step function.

[pseudo-rnd-thoughts]

This looks very impressive, sorry for not responding faster. We are planning on code review and merging your PR, after we have fixed the CI. Would you be interested in working on gym and minigrid more? If so, could you join the gym discord https://discord.gg/nHg2JRN489 and contact me pseudornd