PettingZoo and Stable-Baselines3 example code does not work

[ThomasChen98]

Hi,

When I tried to run python3 sb3_train.py, there were several errors in the original code.

• The env_name here should have two underscores between the environment name and suffix "open", which should be commons_harvest__open https://github.com/deepmind/meltingpot/blob/bb11b0d999483bccdfb8c3488660b63821e42e50/examples/pettingzoo/sb3_train.py#L86

• The __init__ function of the class _MeltingPotPettingZooEnv uses a wrong build function. I believe it should be build_from_config instead of build, and the correct argument should be self._env = substrate.build_from_config(self.env_config,roles=self.env_config.default_player_roles) https://github.com/deepmind/meltingpot/blob/bb11b0d999483bccdfb8c3488660b63821e42e50/examples/pettingzoo/utils.py#L53

Even if I fixed mentioned typos, I still got errors as shown below. Any help on this will be appreciated!

/home/denso/anaconda3/envs/meltingpot/lib/python3.9/site-packages/pettingzoo/utils/conversions.py:196: UserWarning: The base environment _ParallelEnv does not have a render_mode defined. warnings.warn( Traceback (most recent call last): File "/home/denso/yuxin/meltingpot/./examples/pettingzoo/sb3_train.py", line 185, in main() File "/home/denso/yuxin/meltingpot/./examples/pettingzoo/sb3_train.py", line 120, in main env = ss.frame_stack_v1(env, num_frames) File "/home/denso/anaconda3/envs/meltingpot/lib/python3.9/site-packages/supersuit/generic_wrappers/frame_stack.py", line 42, in frame_stack_v1 return shared_wrapper(env, FrameStackModifier) File "/home/denso/anaconda3/envs/meltingpot/lib/python3.9/site-packages/supersuit/utils/wrapper_chooser.py", line 31, in call return self.parallel_wrapper(env, *args, **kwargs) File "/home/denso/anaconda3/envs/meltingpot/lib/python3.9/site-packages/supersuit/generic_wrappers/utils/shared_wrapper_util.py", line 81, in init self.add_modifiers(self.env.possible_agents) File "/home/denso/anaconda3/envs/meltingpot/lib/python3.9/site-packages/supersuit/generic_wrappers/utils/shared_wrapper_util.py", line 96, in add_modifiers self.observation_space(agent) File "/home/denso/anaconda3/envs/meltingpot/lib/python3.9/site-packages/supersuit/generic_wrappers/utils/shared_wrapper_util.py", line 85, in observation_space return self.modifiers[agent].modify_obs_space(self.env.observation_space(agent)) File "/home/denso/anaconda3/envs/meltingpot/lib/python3.9/site-packages/supersuit/generic_wrappers/frame_stack.py", line 19, in modify_obs_space assert ( AssertionError: Stacking is currently only allowed for Box and Discrete observation spaces. The given observation space is Box([[[0 0 0] [0 0 0] [0 0 0] ... [0 0 0] [0 0 0] [0 0 0]] [[0 0 0] [0 0 0] [0 0 0] ... [0 0 0] [0 0 0] [0 0 0]] [[0 0 0] [0 0 0] [0 0 0] ... [0 0 0] [0 0 0] [0 0 0]] ... [[0 0 0] [0 0 0] [0 0 0] ... [0 0 0] [0 0 0] [0 0 0]] [[0 0 0] [0 0 0] [0 0 0] ... [0 0 0] [0 0 0] [0 0 0]] [[0 0 0] [0 0 0] [0 0 0] ... [0 0 0] [0 0 0] [0 0 0]]], [[[255 255 255] [255 255 255] [255 255 255] ... [255 255 255] [255 255 255] [255 255 255]] [[255 255 255] [255 255 255] [255 255 255] ... [255 255 255] [255 255 255] [255 255 255]] [[255 255 255] [255 255 255] [255 255 255] ... [255 255 255] [255 255 255] [255 255 255]] ... [[255 255 255] [255 255 255] [255 255 255] ... [255 255 255] [255 255 255] [255 255 255]] [[255 255 255] [255 255 255] [255 255 255] ... [255 255 255] [255 255 255] [255 255 255]] [[255 255 255] [255 255 255] [255 255 255] ... [255 255 255] [255 255 255] [255 255 255]]], (88, 88, 3), uint8)

[duenez]

Yes, unfortunately we don't use PettingZoo internally, and only just recently we added tests to the (RLLib) examples. It is on my radar to get this updated to Melting Pot 2.0, but if you know Petting Zoo well enough and feel like submitting a pull request with the fix, I'd be happy to review. The changes should be relatively straight-forward. You can base it upon the RLLib change: https://github.com/deepmind/meltingpot/commit/9e473f7eb449ab5dc9335a2fa36dd6fe9fff4f27

[duenez]

Also, parameterising the test like in https://github.com/deepmind/meltingpot/commit/5bb5c48b9ce286f76accc700d344e545a1b863b6 would be good so that it doesn't run for a hard-coded amount of time.

[ThomasChen98]

Yes, unfortunately we don't use PettingZoo internally, and only just recently we added tests to the (RLLib) examples. It is on my radar to get this updated to Melting Pot 2.0, but if you know Petting Zoo well enough and feel like submitting a pull request with the fix, I'd be happy to review. The changes should be relatively straight-forward. You can base it upon the RLLib change: 9e473f7

Has Petting Zoo with SB3 example code been tested? The RLLib examples works fine on my side, but the Petting Zoo example (I believe it is not using RLLib, but Stable Baseline 3 instead) is not functioning at this time. The error messages I received suggested that the Melting Pot wrapper for the Petting Zoo may be broken.

[duenez]

No, the example has not been updated to MeltingPot 2.0, that's why I'm linking the changes on RLLib that updated that example to MP 2.0. The PettingZoo & SB3 example was contributed, so I haven't gotten around to updating it.

[ThomasChen98]

Now I got it. Thanks for your reply!

[theo-michel]

I am curious, what do you use internally, RLLib, or is there another way ?

Yes, unfortunately we don't use PettingZoo internally, and only just recently we added tests to the (RLLib) examples. It is on my radar to get this updated to Melting Pot 2.0, but if you know Petting Zoo well enough and feel like submitting a pull request with the fix, I'd be happy to review. The changes should be relatively straight-forward. You can base it upon the RLLib change: 9e473f7

[duenez]

Internally we have our own distributed training system, custom made for MARL.