Fixing dependencies in `setup.py`

[s-a-barnett]

I have installed Melting Pot via the Dockerfile, and while it passes the tests, I get issues with the RLLib and PettingZoo examples that seem to stem from the versions of a number of packages, and their respective dependencies. I have been able to get the self_play_train.py RLLib example to run using the following combination of packages

pettingzoo==1.18.1
supersuit==3.3.1
stable_baselines3==1.2.0
gym==0.21.0

but the sb3_train.py script currently yields the following error:

Traceback (most recent call last):
  File "/home/samuelab/software/meltingpot/examples/pettingzoo/sb3_train.py", line 185, in <module>
    main()
  File "/home/samuelab/software/meltingpot/examples/pettingzoo/sb3_train.py", line 122, in main
    env = ss.concat_vec_envs_v1(
  File "/home/samuelab/.local/lib/python3.9/site-packages/supersuit/vector/vector_constructors.py", line 49, in concat_vec_envs_v1
    vec_env = MakeCPUAsyncConstructor(num_cpus)(*vec_env_args(vec_env, num_vec_envs))
  File "/home/samuelab/.local/lib/python3.9/site-packages/supersuit/vector/concat_vec_env.py", line 12, in __init__
    self.vec_envs = vec_envs = [vec_env_fn() for vec_env_fn in vec_env_fns]
  File "/home/samuelab/.local/lib/python3.9/site-packages/supersuit/vector/concat_vec_env.py", line 12, in <listcomp>
    self.vec_envs = vec_envs = [vec_env_fn() for vec_env_fn in vec_env_fns]
  File "/home/samuelab/.local/lib/python3.9/site-packages/supersuit/vector/vector_constructors.py", line 11, in env_fn
    env.seed(None)
  File "/home/samuelab/.local/lib/python3.9/site-packages/supersuit/vector/markov_vector_wrapper.py", line 32, in seed
    self.par_env.seed(seed)
  File "/home/samuelab/.local/lib/python3.9/site-packages/supersuit/generic_wrappers/utils/shared_wrapper_util.py", line 89, in seed
    super().seed(seed)
  File "/home/samuelab/.local/lib/python3.9/site-packages/pettingzoo/utils/env.py", line 250, in seed
    raise NotImplementedError(
NotImplementedError: Calling seed externally is deprecated; call reset(seed=seed) instead

Looking into it, this seems to be an issue with supersuit==3.3.1 which is resolved in 3.4.0. But then this version of supersuit requires gym==0.22.0, while stable_baselines3 needs gym<=0.21.0 up until its latest version.

Is there some combination of versions for these packages so that all of the constraints are satisfied and I can get all of the Melting Pot example scripts to run?

[s-a-barnett]

@duenez I have updated Melting Pot to v1.0.4 but this still does not seem to work.

[willis-richard]

Not sure that you want to bump to Ray 2.0.0, the rllib examples were written for ray 1.13.0, and there could be breaking changes.

I think there is some conflict between the sb3 and rllib examples, and the user should only install one of the dependencies.

[duenez]

I've fixed the RLLib example, and updated it to 2.2.0. It should pass at head.