Closed PieroMacaluso closed 2 years ago
For some pretty subtle reasons, when using the env_to_vec_env_v1
, you must follow up by wrapping it in concat_vec_env_v1
.
Thanks for the answer @jjshoots! I tried like this:
def env(**kwargs):
env_ = parallel_env(**kwargs)
env_ = ss.pettingzoo_env_to_vec_env_v1(env_)
env_ = ss.concat_vec_envs_v1(env_, 1)
return env_
petting_zoo = env
class parallel_env(ParallelEnv):
[...]
But I get this error:
cannot pickle 'SwigPyObject' object
You may need to use the EzPickle trick, see the kaz init for an example. Essentially, your base environment class needs to inherit from the EzPickle class.
Thanks! The problem related to pickle is solved, but there is still the initial error.
def env(**kwargs):
env_ = parallel_env(**kwargs)
env_ = ss.pettingzoo_env_to_vec_env_v1(env_)
env_ = ss.concat_vec_envs_v1(env_, 1)
return env_
petting_zoo = env
class parallel_env(ParallelEnv, EzPickle):
[...]
I still get:
The observation returned by the `reset()` method does not match the given observation space
You can probably remove the api check, it's only valid for unwrapped classes. :)
Removing the line with check_env(env)
, the problem is still in the reset function. The error is:
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/home/user/.vscode/extensions/ms-python.python-2022.10.1/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/__main__.py", line 39, in <module>
cli.main()
File "/home/user/.vscode/extensions/ms-python.python-2022.10.1/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 430, in main
run()
File "/home/user/.vscode/extensions/ms-python.python-2022.10.1/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 284, in run_file
runpy.run_path(target, run_name="__main__")
File "/home/user/.vscode/extensions/ms-python.python-2022.10.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 321, in run_path
return _run_module_code(code, init_globals, run_name,
File "/home/user/.vscode/extensions/ms-python.python-2022.10.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 135, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "/home/user/.vscode/extensions/ms-python.python-2022.10.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 124, in _run_code
exec(code, run_globals)
File "/home/user/dev/project/player-env/main_multi.py", line 23, in <module>
model.learn(total_timesteps=10_000)
File "/home/user/dev/project/player-env/.venv/lib/python3.8/site-packages/stable_baselines3/ppo/ppo.py", line 310, in learn
return super().learn(
File "/home/user/dev/project/player-env/.venv/lib/python3.8/site-packages/stable_baselines3/common/on_policy_algorithm.py", line 239, in learn
total_timesteps, callback = self._setup_learn(
File "/home/user/dev/project/player-env/.venv/lib/python3.8/site-packages/stable_baselines3/common/base_class.py", line 446, in _setup_learn
self._last_obs = self.env.reset() # pytype: disable=annotation-type-mismatch
File "/home/user/dev/project/player-env/.venv/lib/python3.8/site-packages/stable_baselines3/common/vec_env/dummy_vec_env.py", line 64, in reset
self._save_obs(env_idx, obs)
File "/home/user/dev/project/player-env/.venv/lib/python3.8/site-packages/stable_baselines3/common/vec_env/dummy_vec_env.py", line 94, in _save_obs
self.buf_obs[key][env_idx] = obs
ValueError: could not broadcast input array from shape (20,20) into shape (20,)
The reset()
function is still returning an array which is the concatenation of the observation for all the players (20).
Would if be ok for you to send me the files that you're working on? I'll have a poke around and see what I can find out. I think that is more productive than iterative bug squashing over this issues channel.
I agree @jjshoots, thanks again for your time! I've just prepared a dummy repo with a very simple environment to reproduce the error. You can find it here: https://github.com/PieroMacaluso/dummy-env
Hi @PieroMacaluso, apologies for the delay, I finally got the time today to look at this.
I believe the fix to your problem is this:
base_class="stable_baselines3"
to concat_vec_envs_v1
, apologies for not stating this earlier.That said, the correct wrapper order for your environment should be:
env_parallel = dummy.parallel_env()
parallel_api_test(env_parallel) # This works!
env_parallel = ss.pettingzoo_env_to_vec_env_v1(env_parallel)
env_parallel = ss.concat_vec_envs_v1(env_parallel, 1, base_class="stable_baselines3")
Let me know if you encounter any other issues. :)
Hi @jjshoots! Finally good news! :partying_face: I followed your guidelines and these are the outcomes.
check_env(env_parallel)
I got an error AssertionError: Your environment must inherit from the gym.Env class cf https://github.com/openai/gym/blob/master/gym/core.py
check_env(env_parallel)
, the agent starts training successfully!In the end, I think that the argument base_class="stable_baselines3"
made the difference.
Only the small problem on check_env
remains to be reported, but I think it can be considered as trivial if the training works.
Thanks a lot for your help and support!
I am having trouble in making things work with a Custom
ParallelEnv
I wrote by using PettingZoo. I am using SuperSuit'sss.pettingzoo_env_to_vec_env_v1(env)
as a wrapper to Vectorize the environment and make it work with Stable-Baseline3 as documented here.You can find attached a summary of the most relevant part of the code:
Unfortunately when I try to test with the following main procedure:
I get the following error:
It seems like that
ss.pettingzoo_env_to_vec_env_v1(env)
is capable of splitting the parallel environment in multiple vectorized ones, but not for thereset()
function.Does anyone know how to fix this problem?