Gym 0.25.0 incompatible with d4rl

[juliusfrost]

The recent 0.25.0 gym release introduces returning 5 variables from the step function, which d4rl is not updated to handle. This PR enforces gym<0.25.0 until d4rl is updated. As a bonus, it sets dm_control>=1.0.3 which is the latest release instead of master which can break downstream installations.

[juliusfrost]

@justinjfu would very much appreciate it if you could merge this.

[im-Kitsch]

Hi,

I checked gym 0.25, looks it still returns 4 variables, just run this

env = gym.make('Ant-v2')
env.reset()
env.step(env.action_space.sample())

It still returns 4 states since by gym.make(new_step_api=False), so it still uses old apis and compatible for d4rl.

I emphasize this because it shows that gym0.24 and 0.24.1 is highly unrecommended in website. There is explicit warning when you import gym of 0.24/0.24.1:

Warning: Gym version v0.24.1 has a number of critical issues with gym.make such that environment observation and action spaces are incorrectly evaluated, raising incorrect errors and warning . It is recommend to downgrading to v0.23.1 or upgrading to v0.25.1

While in requirements.txt it requires gym<0.25, it will automatically change gym to 0.24.1, it look dangerous to use 0.24.1.

I think it would be better to change gym<0.25.0 to gym<=0.25.1? Otherwise it would be safe to change as gym<0.24.0

Best!