LucasAlegre
commented
1 year ago Hi @Unamed318 , what version of Rllib and Gymnasium are you using? please update all your libraries and try again.
Closed Unamed318 closed 1 year ago
LucasAlegre
commented
1 year ago Hi @Unamed318 , what version of Rllib and Gymnasium are you using? please update all your libraries and try again.
Unamed318
commented
1 year ago After I have updated all my libraries according to this webpage: "https://www.activestate.com/resources/quick-reads/how-to-update-all-python-packages/", there is now another error
(EE) root@3sec3sec3sec:/home/sam/Documents/EE/sumo-rl-main_new/experiments# python ppo_4x4grid.py
2023-07-01 17:55:21,139 INFO worker.py:1636 -- Started a local Ray instance.
/usr/local/lib/python3.10/dist-packages/ray/tune/tune.py:258: UserWarning: Passing a `local_dir` is deprecated and will be removed in the future. Pass `storage_path` instead or set the`RAY_AIR_LOCAL_CACHE_DIR` environment variable instead.
warnings.warn(
2023-07-01 17:55:21,662 WARNING deprecation.py:50 -- DeprecationWarning: `rllib/algorithms/a3c/a3c.py` has been deprecated. Use `rllib_contrib/a3c/` instead. This will raise an error in the future!
2023-07-01 17:55:21,662 WARNING deprecation.py:50 -- DeprecationWarning: `rllib/algorithms/a3c/a3c.py` has been deprecated. Use `rllib_contrib/a3c/` instead. This will raise an error in the future!
2023-07-01 17:55:21,691 WARNING deprecation.py:50 -- DeprecationWarning: `rllib/algorithms/maml/maml.py` has been deprecated. Use `rllib_contrib/maml/` instead. This will raise an error in the future!
== Status ==
Current time: 2023-07-01 17:55:21 (running for 00:00:00.11)
Using FIFO scheduling algorithm.
Logical resource usage: 5.0/24 CPUs, 0/0 GPUs
Result logdir: /root/ray_results/4x4grid/PPO
Number of trials: 1/1 (1 PENDING)
+-------------------------+----------+-------+
| Trial name | status | loc |
|-------------------------+----------+-------|
| PPO_4x4grid_64959_00000 | PENDING | |
+-------------------------+----------+-------+
(PPO pid=252374) 2023-07-01 17:55:24,203 WARNING algorithm_config.py:643 -- Cannot create PPOConfig from given `config_dict`! Property __stdout_file__ not supported.
(PPO pid=252374) 2023-07-01 17:55:24,216 INFO algorithm.py:536 -- Current log_level is ERROR. For more information, set 'log_level': 'INFO' / 'DEBUG' or use the -v and -vv flags.
== Status ==
Current time: 2023-07-01 17:55:26 (running for 00:00:05.12)
Using FIFO scheduling algorithm.
Logical resource usage: 5.0/24 CPUs, 0/0 GPUs
Result logdir: /root/ray_results/4x4grid/PPO
Number of trials: 1/1 (1 PENDING)
+-------------------------+----------+-------+
| Trial name | status | loc |
|-------------------------+----------+-------|
| PPO_4x4grid_64959_00000 | PENDING | |
+-------------------------+----------+-------+
(RolloutWorker pid=252438) Retrying in 1 seconds
(RolloutWorker pid=252435) Step #0.00 (0ms ?*RT. ?UPS, TraCI: 8ms, vehicles TOT 0 ACT 0 BUF 0)
(RolloutWorker pid=252435) 2023-07-01 17:55:28,887 ERROR worker.py:861 -- Exception raised in creation task: The actor died because of an error raised in its creation task, ray::RolloutWorker.__init__() (pid=252435, ip=192.168.50.3, actor_id=3221ab97da6d8ecc8d8c743e01000000, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7f016de792d0>)
(RolloutWorker pid=252435) File "/usr/local/lib/python3.10/dist-packages/ray/rllib/env/wrappers/pettingzoo_env.py", line 224, in reset
(RolloutWorker pid=252435) obs, info = self.par_env.reset(seed=seed, return_info=True, options=options)
(RolloutWorker pid=252435) TypeError: aec_to_parallel_wrapper.reset() got an unexpected keyword argument 'return_info'
(RolloutWorker pid=252435)
(RolloutWorker pid=252435) The above exception was the direct cause of the following exception:
(RolloutWorker pid=252435)
(RolloutWorker pid=252435) ray::RolloutWorker.__init__() (pid=252435, ip=192.168.50.3, actor_id=3221ab97da6d8ecc8d8c743e01000000, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7f016de792d0>)
(RolloutWorker pid=252435) File "/usr/local/lib/python3.10/dist-packages/ray/rllib/utils/pre_checks/env.py", line 77, in check_env
(RolloutWorker pid=252435) check_multiagent_environments(env)
(RolloutWorker pid=252435) File "/usr/local/lib/python3.10/dist-packages/ray/rllib/utils/pre_checks/env.py", line 299, in check_multiagent_environments
(RolloutWorker pid=252435) raise ValueError(
(RolloutWorker pid=252435) ValueError: Your environment (<ParallelPettingZooEnv instance>) does not abide to the new gymnasium-style API!
(RolloutWorker pid=252435) From Ray 2.3 on, RLlib only supports the new (gym>=0.26 or gymnasium) Env APIs.
(RolloutWorker pid=252435) In particular, the `reset()` method seems to be faulty.
(RolloutWorker pid=252435) Learn more about the most important changes here:
(RolloutWorker pid=252435) https://github.com/openai/gym and here: https://github.com/Farama-Foundation/Gymnasium
(RolloutWorker pid=252435)
(RolloutWorker pid=252435) In order to fix this problem, do the following:
(RolloutWorker pid=252435)
(RolloutWorker pid=252435) 1) Run `pip install gymnasium` on your command line.
(RolloutWorker pid=252435) 2) Change all your import statements in your code from
(RolloutWorker pid=252435) `import gym` -> `import gymnasium as gym` OR
(RolloutWorker pid=252435) `from gym.space import Discrete` -> `from gymnasium.spaces import Discrete`
(RolloutWorker pid=252435)
(RolloutWorker pid=252435) For your custom (single agent) gym.Env classes:
(RolloutWorker pid=252435) 3.1) Either wrap your old Env class via the provided `from gymnasium.wrappers import
(RolloutWorker pid=252435) EnvCompatibility` wrapper class.
(RolloutWorker pid=252435) 3.2) Alternatively to 3.1:
(RolloutWorker pid=252435) - Change your `reset()` method to have the call signature 'def reset(self, *,
(RolloutWorker pid=252435) seed=None, options=None)'
(RolloutWorker pid=252435) - Return an additional info dict (empty dict should be fine) from your `reset()`
(RolloutWorker pid=252435) method.
(RolloutWorker pid=252435) - Return an additional `truncated` flag from your `step()` method (between `done` and
(RolloutWorker pid=252435) `info`). This flag should indicate, whether the episode was terminated prematurely
(RolloutWorker pid=252435) due to some time constraint or other kind of horizon setting.
(RolloutWorker pid=252435)
(RolloutWorker pid=252435) For your custom RLlib `MultiAgentEnv` classes:
(RolloutWorker pid=252435) 4.1) Either wrap your old MultiAgentEnv via the provided
(RolloutWorker pid=252435) `from ray.rllib.env.wrappers.multi_agent_env_compatibility import
(RolloutWorker pid=252435) MultiAgentEnvCompatibility` wrapper class.
(RolloutWorker pid=252435) 4.2) Alternatively to 4.1:
(RolloutWorker pid=252435) - Change your `reset()` method to have the call signature
(RolloutWorker pid=252435) 'def reset(self, *, seed=None, options=None)'
(RolloutWorker pid=252435) - Return an additional per-agent info dict (empty dict should be fine) from your
(RolloutWorker pid=252435) `reset()` method.
(RolloutWorker pid=252435) - Rename `dones` into `terminateds` and only set this to True, if the episode is really
(RolloutWorker pid=252435) done (as opposed to has been terminated prematurely due to some horizon/time-limit
(RolloutWorker pid=252435) setting).
(RolloutWorker pid=252435) - Return an additional `truncateds` per-agent dictionary flag from your `step()`
(RolloutWorker pid=252435) method, including the `__all__` key (100% analogous to your `dones/terminateds`
(RolloutWorker pid=252435) per-agent dict).
(RolloutWorker pid=252435) Return this new `truncateds` dict between `dones/terminateds` and `infos`. This
(RolloutWorker pid=252435) flag should indicate, whether the episode (for some agent or all agents) was
(RolloutWorker pid=252435) terminated prematurely due to some time constraint or other kind of horizon setting.
(RolloutWorker pid=252435)
(RolloutWorker pid=252435)
(RolloutWorker pid=252435) During handling of the above exception, another exception occurred:
(RolloutWorker pid=252435)
(RolloutWorker pid=252435) ray::RolloutWorker.__init__() (pid=252435, ip=192.168.50.3, actor_id=3221ab97da6d8ecc8d8c743e01000000, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7f016de792d0>)
(RolloutWorker pid=252435) File "/usr/local/lib/python3.10/dist-packages/ray/rllib/evaluation/rollout_worker.py", line 616, in __init__
(RolloutWorker pid=252435) check_env(self.env)
(RolloutWorker pid=252435) File "/usr/local/lib/python3.10/dist-packages/ray/rllib/utils/pre_checks/env.py", line 92, in check_env
(RolloutWorker pid=252435) raise ValueError(
(RolloutWorker pid=252435) ValueError: Traceback (most recent call last):
(RolloutWorker pid=252435) File "/usr/local/lib/python3.10/dist-packages/ray/rllib/utils/pre_checks/env.py", line 294, in check_multiagent_environments
(RolloutWorker pid=252435) obs_and_infos = env.reset(seed=42, options={})
(RolloutWorker pid=252435) File "/usr/local/lib/python3.10/dist-packages/ray/rllib/env/wrappers/pettingzoo_env.py", line 224, in reset
(RolloutWorker pid=252435) obs, info = self.par_env.reset(seed=seed, return_info=True, options=options)
(RolloutWorker pid=252435) TypeError: aec_to_parallel_wrapper.reset() got an unexpected keyword argument 'return_info'
(RolloutWorker pid=252435)
(RolloutWorker pid=252435) The above exception was the direct cause of the following exception:
(RolloutWorker pid=252435)
(RolloutWorker pid=252435) ray::RolloutWorker.__init__() (pid=252435, ip=192.168.50.3, actor_id=3221ab97da6d8ecc8d8c743e01000000, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7f016de792d0>)
(RolloutWorker pid=252435) File "/usr/local/lib/python3.10/dist-packages/ray/rllib/utils/pre_checks/env.py", line 77, in check_env
(RolloutWorker pid=252435) check_multiagent_environments(env)
(RolloutWorker pid=252435) File "/usr/local/lib/python3.10/dist-packages/ray/rllib/utils/pre_checks/env.py", line 299, in check_multiagent_environments
(RolloutWorker pid=252435) raise ValueError(
(RolloutWorker pid=252435) ValueError: Your environment (<ParallelPettingZooEnv instance>) does not abide to the new gymnasium-style API!
(RolloutWorker pid=252435) From Ray 2.3 on, RLlib only supports the new (gym>=0.26 or gymnasium) Env APIs.
(RolloutWorker pid=252435) In particular, the `reset()` method seems to be faulty.
(RolloutWorker pid=252435) Learn more about the most important changes here:
(RolloutWorker pid=252435) https://github.com/openai/gym and here: https://github.com/Farama-Foundation/Gymnasium
(RolloutWorker pid=252435)
(RolloutWorker pid=252435) In order to fix this problem, do the following:
(RolloutWorker pid=252435)
(RolloutWorker pid=252435) 1) Run `pip install gymnasium` on your command line.
(RolloutWorker pid=252435) 2) Change all your import statements in your code from
(RolloutWorker pid=252435) `import gym` -> `import gymnasium as gym` OR
(RolloutWorker pid=252435) `from gym.space import Discrete` -> `from gymnasium.spaces import Discrete`
(RolloutWorker pid=252435)
(RolloutWorker pid=252435) For your custom (single agent) gym.Env classes:
(RolloutWorker pid=252435) 3.1) Either wrap your old Env class via the provided `from gymnasium.wrappers import
(RolloutWorker pid=252435) EnvCompatibility` wrapper class.
(RolloutWorker pid=252435) 3.2) Alternatively to 3.1:
(RolloutWorker pid=252435) - Change your `reset()` method to have the call signature 'def reset(self, *,
(RolloutWorker pid=252435) seed=None, options=None)'
(RolloutWorker pid=252435) - Return an additional info dict (empty dict should be fine) from your `reset()`
(RolloutWorker pid=252435) method.
(RolloutWorker pid=252435) - Return an additional `truncated` flag from your `step()` method (between `done` and
(RolloutWorker pid=252435) `info`). This flag should indicate, whether the episode was terminated prematurely
(RolloutWorker pid=252435) due to some time constraint or other kind of horizon setting.
(RolloutWorker pid=252435)
(RolloutWorker pid=252435) For your custom RLlib `MultiAgentEnv` classes:
(RolloutWorker pid=252435) 4.1) Either wrap your old MultiAgentEnv via the provided
(RolloutWorker pid=252435) `from ray.rllib.env.wrappers.multi_agent_env_compatibility import
(RolloutWorker pid=252435) MultiAgentEnvCompatibility` wrapper class.
(RolloutWorker pid=252435) 4.2) Alternatively to 4.1:
(RolloutWorker pid=252435) - Change your `reset()` method to have the call signature
(RolloutWorker pid=252435) 'def reset(self, *, seed=None, options=None)'
(RolloutWorker pid=252435) - Return an additional per-agent info dict (empty dict should be fine) from your
(RolloutWorker pid=252435) `reset()` method.
(RolloutWorker pid=252435) - Rename `dones` into `terminateds` and only set this to True, if the episode is really
(RolloutWorker pid=252435) done (as opposed to has been terminated prematurely due to some horizon/time-limit
(RolloutWorker pid=252435) setting).
(RolloutWorker pid=252435) - Return an additional `truncateds` per-agent dictionary flag from your `step()`
(RolloutWorker pid=252435) method, including the `__all__` key (100% analogous to your `dones/terminateds`
(RolloutWorker pid=252435) per-agent dict).
(RolloutWorker pid=252435) Return this new `truncateds` dict between `dones/terminateds` and `infos`. This
(RolloutWorker pid=252435) flag should indicate, whether the episode (for some agent or all agents) was
(RolloutWorker pid=252435) terminated prematurely due to some time constraint or other kind of horizon setting.
(RolloutWorker pid=252435)
(RolloutWorker pid=252435)
(RolloutWorker pid=252435) The above error has been found in your environment! We've added a module for checking your custom environments. It may cause your experiment to fail if your environment is not set up correctly. You can disable this behavior via calling `config.environment(disable_env_checking=True)`. You can run the environment checking module standalone by calling ray.rllib.utils.check_env([your env]).
2023-07-01 17:55:29,031 ERROR tune_controller.py:873 -- Trial task failed for trial PPO_4x4grid_64959_00000
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/ray/air/execution/_internal/event_manager.py", line 110, in resolve_future
result = ray.get(future)
File "/usr/local/lib/python3.10/dist-packages/ray/_private/auto_init_hook.py", line 18, in auto_init_wrapper
return fn(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/ray/_private/client_mode_hook.py", line 103, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/ray/_private/worker.py", line 2542, in get
raise value
ray.exceptions.RayActorError: The actor died because of an error raised in its creation task, ray::PPO.__init__() (pid=252374, ip=192.168.50.3, actor_id=b35b6c13f241014cede19ba701000000, repr=PPO)
File "/usr/local/lib/python3.10/dist-packages/ray/rllib/evaluation/worker_set.py", line 242, in _setup
self.add_workers(
File "/usr/local/lib/python3.10/dist-packages/ray/rllib/evaluation/worker_set.py", line 635, in add_workers
raise result.get()
File "/usr/local/lib/python3.10/dist-packages/ray/rllib/utils/actor_manager.py", line 488, in __fetch_result
result = ray.get(r)
ray.exceptions.RayActorError: The actor died because of an error raised in its creation task, ray::RolloutWorker.__init__() (pid=252435, ip=192.168.50.3, actor_id=3221ab97da6d8ecc8d8c743e01000000, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7f016de792d0>)
File "/usr/local/lib/python3.10/dist-packages/ray/rllib/env/wrappers/pettingzoo_env.py", line 224, in reset
obs, info = self.par_env.reset(seed=seed, return_info=True, options=options)
TypeError: aec_to_parallel_wrapper.reset() got an unexpected keyword argument 'return_info'
The above exception was the direct cause of the following exception:
ray::RolloutWorker.__init__() (pid=252435, ip=192.168.50.3, actor_id=3221ab97da6d8ecc8d8c743e01000000, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7f016de792d0>)
File "/usr/local/lib/python3.10/dist-packages/ray/rllib/utils/pre_checks/env.py", line 77, in check_env
check_multiagent_environments(env)
File "/usr/local/lib/python3.10/dist-packages/ray/rllib/utils/pre_checks/env.py", line 299, in check_multiagent_environments
raise ValueError(
ValueError: Your environment (<ParallelPettingZooEnv instance>) does not abide to the new gymnasium-style API!
From Ray 2.3 on, RLlib only supports the new (gym>=0.26 or gymnasium) Env APIs.
In particular, the `reset()` method seems to be faulty.
Learn more about the most important changes here:
https://github.com/openai/gym and here: https://github.com/Farama-Foundation/Gymnasium
In order to fix this problem, do the following:
1) Run `pip install gymnasium` on your command line.
2) Change all your import statements in your code from
`import gym` -> `import gymnasium as gym` OR
`from gym.space import Discrete` -> `from gymnasium.spaces import Discrete`
For your custom (single agent) gym.Env classes:
3.1) Either wrap your old Env class via the provided `from gymnasium.wrappers import
EnvCompatibility` wrapper class.
3.2) Alternatively to 3.1:
- Change your `reset()` method to have the call signature 'def reset(self, *,
seed=None, options=None)'
- Return an additional info dict (empty dict should be fine) from your `reset()`
method.
- Return an additional `truncated` flag from your `step()` method (between `done` and
`info`). This flag should indicate, whether the episode was terminated prematurely
due to some time constraint or other kind of horizon setting.
For your custom RLlib `MultiAgentEnv` classes:
4.1) Either wrap your old MultiAgentEnv via the provided
`from ray.rllib.env.wrappers.multi_agent_env_compatibility import
MultiAgentEnvCompatibility` wrapper class.
4.2) Alternatively to 4.1:
- Change your `reset()` method to have the call signature
'def reset(self, *, seed=None, options=None)'
- Return an additional per-agent info dict (empty dict should be fine) from your
`reset()` method.
- Rename `dones` into `terminateds` and only set this to True, if the episode is really
done (as opposed to has been terminated prematurely due to some horizon/time-limit
setting).
- Return an additional `truncateds` per-agent dictionary flag from your `step()`
method, including the `__all__` key (100% analogous to your `dones/terminateds`
per-agent dict).
Return this new `truncateds` dict between `dones/terminateds` and `infos`. This
flag should indicate, whether the episode (for some agent or all agents) was
terminated prematurely due to some time constraint or other kind of horizon setting.
During handling of the above exception, another exception occurred:
ray::RolloutWorker.__init__() (pid=252435, ip=192.168.50.3, actor_id=3221ab97da6d8ecc8d8c743e01000000, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7f016de792d0>)
File "/usr/local/lib/python3.10/dist-packages/ray/rllib/evaluation/rollout_worker.py", line 616, in __init__
check_env(self.env)
File "/usr/local/lib/python3.10/dist-packages/ray/rllib/utils/pre_checks/env.py", line 92, in check_env
raise ValueError(
ValueError: Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/ray/rllib/utils/pre_checks/env.py", line 294, in check_multiagent_environments
obs_and_infos = env.reset(seed=42, options={})
File "/usr/local/lib/python3.10/dist-packages/ray/rllib/env/wrappers/pettingzoo_env.py", line 224, in reset
obs, info = self.par_env.reset(seed=seed, return_info=True, options=options)
TypeError: aec_to_parallel_wrapper.reset() got an unexpected keyword argument 'return_info'
The above exception was the direct cause of the following exception:
ray::RolloutWorker.__init__() (pid=252435, ip=192.168.50.3, actor_id=3221ab97da6d8ecc8d8c743e01000000, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7f016de792d0>)
File "/usr/local/lib/python3.10/dist-packages/ray/rllib/utils/pre_checks/env.py", line 77, in check_env
check_multiagent_environments(env)
File "/usr/local/lib/python3.10/dist-packages/ray/rllib/utils/pre_checks/env.py", line 299, in check_multiagent_environments
raise ValueError(
ValueError: Your environment (<ParallelPettingZooEnv instance>) does not abide to the new gymnasium-style API!
From Ray 2.3 on, RLlib only supports the new (gym>=0.26 or gymnasium) Env APIs.
In particular, the `reset()` method seems to be faulty.
Learn more about the most important changes here:
https://github.com/openai/gym and here: https://github.com/Farama-Foundation/Gymnasium
In order to fix this problem, do the following:
1) Run `pip install gymnasium` on your command line.
2) Change all your import statements in your code from
`import gym` -> `import gymnasium as gym` OR
`from gym.space import Discrete` -> `from gymnasium.spaces import Discrete`
For your custom (single agent) gym.Env classes:
3.1) Either wrap your old Env class via the provided `from gymnasium.wrappers import
EnvCompatibility` wrapper class.
3.2) Alternatively to 3.1:
- Change your `reset()` method to have the call signature 'def reset(self, *,
seed=None, options=None)'
- Return an additional info dict (empty dict should be fine) from your `reset()`
method.
- Return an additional `truncated` flag from your `step()` method (between `done` and
`info`). This flag should indicate, whether the episode was terminated prematurely
due to some time constraint or other kind of horizon setting.
For your custom RLlib `MultiAgentEnv` classes:
4.1) Either wrap your old MultiAgentEnv via the provided
`from ray.rllib.env.wrappers.multi_agent_env_compatibility import
MultiAgentEnvCompatibility` wrapper class.
4.2) Alternatively to 4.1:
- Change your `reset()` method to have the call signature
'def reset(self, *, seed=None, options=None)'
- Return an additional per-agent info dict (empty dict should be fine) from your
`reset()` method.
- Rename `dones` into `terminateds` and only set this to True, if the episode is really
done (as opposed to has been terminated prematurely due to some horizon/time-limit
setting).
- Return an additional `truncateds` per-agent dictionary flag from your `step()`
method, including the `__all__` key (100% analogous to your `dones/terminateds`
per-agent dict).
Return this new `truncateds` dict between `dones/terminateds` and `infos`. This
flag should indicate, whether the episode (for some agent or all agents) was
terminated prematurely due to some time constraint or other kind of horizon setting.
The above error has been found in your environment! We've added a module for checking your custom environments. It may cause your experiment to fail if your environment is not set up correctly. You can disable this behavior via calling `config.environment(disable_env_checking=True)`. You can run the environment checking module standalone by calling ray.rllib.utils.check_env([your env]).
During handling of the above exception, another exception occurred:
ray::PPO.__init__() (pid=252374, ip=192.168.50.3, actor_id=b35b6c13f241014cede19ba701000000, repr=PPO)
File "/usr/local/lib/python3.10/dist-packages/ray/rllib/algorithms/algorithm.py", line 475, in __init__
super().__init__(
File "/usr/local/lib/python3.10/dist-packages/ray/tune/trainable/trainable.py", line 170, in __init__
self.setup(copy.deepcopy(self.config))
File "/usr/local/lib/python3.10/dist-packages/ray/rllib/algorithms/algorithm.py", line 601, in setup
self.workers = WorkerSet(
File "/usr/local/lib/python3.10/dist-packages/ray/rllib/evaluation/worker_set.py", line 194, in __init__
raise e.args[0].args[2]
ValueError: Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/ray/rllib/utils/pre_checks/env.py", line 294, in check_multiagent_environments
obs_and_infos = env.reset(seed=42, options={})
File "/usr/local/lib/python3.10/dist-packages/ray/rllib/env/wrappers/pettingzoo_env.py", line 224, in reset
obs, info = self.par_env.reset(seed=seed, return_info=True, options=options)
TypeError: aec_to_parallel_wrapper.reset() got an unexpected keyword argument 'return_info'
The above exception was the direct cause of the following exception:
ray::PPO.__init__() (pid=252374, ip=192.168.50.3, actor_id=b35b6c13f241014cede19ba701000000, repr=PPO)
File "/usr/local/lib/python3.10/dist-packages/ray/rllib/utils/pre_checks/env.py", line 77, in check_env
check_multiagent_environments(env)
File "/usr/local/lib/python3.10/dist-packages/ray/rllib/utils/pre_checks/env.py", line 299, in check_multiagent_environments
raise ValueError(
ValueError: Your environment (<ParallelPettingZooEnv instance>) does not abide to the new gymnasium-style API!
From Ray 2.3 on, RLlib only supports the new (gym>=0.26 or gymnasium) Env APIs.
In particular, the `reset()` method seems to be faulty.
Learn more about the most important changes here:
https://github.com/openai/gym and here: https://github.com/Farama-Foundation/Gymnasium
In order to fix this problem, do the following:
1) Run `pip install gymnasium` on your command line.
2) Change all your import statements in your code from
`import gym` -> `import gymnasium as gym` OR
`from gym.space import Discrete` -> `from gymnasium.spaces import Discrete`
For your custom (single agent) gym.Env classes:
3.1) Either wrap your old Env class via the provided `from gymnasium.wrappers import
EnvCompatibility` wrapper class.
3.2) Alternatively to 3.1:
- Change your `reset()` method to have the call signature 'def reset(self, *,
seed=None, options=None)'
- Return an additional info dict (empty dict should be fine) from your `reset()`
method.
- Return an additional `truncated` flag from your `step()` method (between `done` and
`info`). This flag should indicate, whether the episode was terminated prematurely
due to some time constraint or other kind of horizon setting.
For your custom RLlib `MultiAgentEnv` classes:
4.1) Either wrap your old MultiAgentEnv via the provided
`from ray.rllib.env.wrappers.multi_agent_env_compatibility import
MultiAgentEnvCompatibility` wrapper class.
4.2) Alternatively to 4.1:
- Change your `reset()` method to have the call signature
'def reset(self, *, seed=None, options=None)'
- Return an additional per-agent info dict (empty dict should be fine) from your
`reset()` method.
- Rename `dones` into `terminateds` and only set this to True, if the episode is really
done (as opposed to has been terminated prematurely due to some horizon/time-limit
setting).
- Return an additional `truncateds` per-agent dictionary flag from your `step()`
method, including the `__all__` key (100% analogous to your `dones/terminateds`
per-agent dict).
Return this new `truncateds` dict between `dones/terminateds` and `infos`. This
flag should indicate, whether the episode (for some agent or all agents) was
terminated prematurely due to some time constraint or other kind of horizon setting.
The above error has been found in your environment! We've added a module for checking your custom environments. It may cause your experiment to fail if your environment is not set up correctly. You can disable this behavior via calling `config.environment(disable_env_checking=True)`. You can run the environment checking module standalone by calling ray.rllib.utils.check_env([your env]).
Result for PPO_4x4grid_64959_00000:
trial_id: '64959_00000'
== Status ==
Current time: 2023-07-01 17:55:29 (running for 00:00:07.33)
Using FIFO scheduling algorithm.
Logical resource usage: 0/24 CPUs, 0/0 GPUs
Result logdir: /root/ray_results/4x4grid/PPO
Number of trials: 1/1 (1 ERROR)
+-------------------------+----------+-------+
| Trial name | status | loc |
|-------------------------+----------+-------|
| PPO_4x4grid_64959_00000 | ERROR | |
+-------------------------+----------+-------+
Number of errored trials: 1
+-------------------------+--------------+---------------------------------------------------------------------------------------+
| Trial name | # failures | error file |
|-------------------------+--------------+---------------------------------------------------------------------------------------|
| PPO_4x4grid_64959_00000 | 1 | /root/ray_results/4x4grid/PPO/PPO_4x4grid_64959_00000_0_2023-07-01_17-55-21/error.txt |
+-------------------------+--------------+---------------------------------------------------------------------------------------+
Traceback (most recent call last):
File "/home/sam/Documents/EE/sumo-rl-main_new/experiments/ppo_4x4grid.py", line 62, in <module>
tune.run(
File "/usr/local/lib/python3.10/dist-packages/ray/tune/tune.py", line 1105, in run
raise TuneError("Trials did not complete", incomplete_trials)
ray.tune.error.TuneError: ('Trials did not complete', [PPO_4x4grid_64959_00000])
(PPO pid=252374) 2023-07-01 17:55:29,022 ERROR actor_manager.py:507 -- Ray error, taking actor 1 out of service. The actor died because of an error raised in its creation task, ray::RolloutWorker.__init__() (pid=252435, ip=192.168.50.3, actor_id=3221ab97da6d8ecc8d8c743e01000000, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7f016de792d0>)
(PPO pid=252374) 2023-07-01 17:55:29,022 ERROR actor_manager.py:507 -- Ray error, taking actor 2 out of service. The actor died because of an error raised in its creation task, ray::RolloutWorker.__init__() (pid=252436, ip=192.168.50.3, actor_id=9233ed13ef42e21ff618e3df01000000, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7efabd571300>)
(PPO pid=252374) 2023-07-01 17:55:29,022 ERROR actor_manager.py:507 -- Ray error, taking actor 3 out of service. The actor died because of an error raised in its creation task, ray::RolloutWorker.__init__() (pid=252437, ip=192.168.50.3, actor_id=72a5c73ecdc4d5e8a4b3c8ed01000000, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7f8eff1a1360>)
(PPO pid=252374) 2023-07-01 17:55:29,022 ERROR actor_manager.py:507 -- Ray error, taking actor 4 out of service. The actor died because of an error raised in its creation task, ray::RolloutWorker.__init__() (pid=252438, ip=192.168.50.3, actor_id=dd3083fd293adc2d791642a901000000, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7fe12ff91330>)
(PPO pid=252374) 2023-07-01 17:55:29,024 ERROR worker.py:861 -- Exception raised in creation task: The actor died because of an error raised in its creation task, ray::PPO.__init__() (pid=252374, ip=192.168.50.3, actor_id=b35b6c13f241014cede19ba701000000, repr=PPO)
(PPO pid=252374) File "/usr/local/lib/python3.10/dist-packages/ray/rllib/evaluation/worker_set.py", line 242, in _setup
(PPO pid=252374) self.add_workers(
(PPO pid=252374) File "/usr/local/lib/python3.10/dist-packages/ray/rllib/evaluation/worker_set.py", line 635, in add_workers
(PPO pid=252374) raise result.get()
(PPO pid=252374) File "/usr/local/lib/python3.10/dist-packages/ray/rllib/utils/actor_manager.py", line 488, in __fetch_result
(PPO pid=252374) result = ray.get(r)
(PPO pid=252374) ray.exceptions.RayActorError: The actor died because of an error raised in its creation task, ray::RolloutWorker.__init__() (pid=252435, ip=192.168.50.3, actor_id=3221ab97da6d8ecc8d8c743e01000000, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7f016de792d0>)
(PPO pid=252374) ray::PPO.__init__() (pid=252374, ip=192.168.50.3, actor_id=b35b6c13f241014cede19ba701000000, repr=PPO)
(PPO pid=252374) super().__init__(
(PPO pid=252374) self.setup(copy.deepcopy(self.config))
(PPO pid=252374) File "/usr/local/lib/python3.10/dist-packages/ray/rllib/algorithms/algorithm.py", line 601, in setup
(PPO pid=252374) self.workers = WorkerSet(
(PPO pid=252374) raise e.args[0].args[2]
(PPO pid=252374) ray::PPO.__init__() (pid=252374, ip=192.168.50.3, actor_id=b35b6c13f241014cede19ba701000000, repr=PPO)
(RolloutWorker pid=252437) Retrying in 1 seconds [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/ray-logging.html#log-deduplication for more options.)
(RolloutWorker pid=252437) Step #0.00 (0ms ?*RT. ?UPS, TraCI: 8ms, vehicles TOT 0 ACT 0 BUF 0) [repeated 3x across cluster]
(RolloutWorker pid=252437) 2023-07-01 17:55:29,014 ERROR worker.py:861 -- Exception raised in creation task: The actor died because of an error raised in its creation task, ray::RolloutWorker.__init__() (pid=252437, ip=192.168.50.3, actor_id=72a5c73ecdc4d5e8a4b3c8ed01000000, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7f8eff1a1360>) [repeated 3x across cluster]
(RolloutWorker pid=252437) File "/usr/local/lib/python3.10/dist-packages/ray/rllib/env/wrappers/pettingzoo_env.py", line 224, in reset [repeated 17x across cluster]
(RolloutWorker pid=252437) obs, info = self.par_env.reset(seed=seed, return_info=True, options=options) [repeated 17x across cluster]
(RolloutWorker pid=252437) TypeError: aec_to_parallel_wrapper.reset() got an unexpected keyword argument 'return_info' [repeated 17x across cluster]
(RolloutWorker pid=252437) [repeated 146x across cluster]
(RolloutWorker pid=252437) The above exception was the direct cause of the following exception: [repeated 17x across cluster]
(RolloutWorker pid=252437) ray::RolloutWorker.__init__() (pid=252437, ip=192.168.50.3, actor_id=72a5c73ecdc4d5e8a4b3c8ed01000000, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7f8eff1a1360>) [repeated 24x across cluster]
(RolloutWorker pid=252437) File "/usr/local/lib/python3.10/dist-packages/ray/rllib/utils/pre_checks/env.py", line 77, in check_env [repeated 25x across cluster]
(RolloutWorker pid=252437) check_multiagent_environments(env) [repeated 17x across cluster]
(RolloutWorker pid=252437) File "/usr/local/lib/python3.10/dist-packages/ray/rllib/utils/pre_checks/env.py", line 299, in check_multiagent_environments [repeated 26x across cluster]
(RolloutWorker pid=252437) raise ValueError( [repeated 25x across cluster]
(RolloutWorker pid=252437) ValueError: Your environment (<ParallelPettingZooEnv instance>) does not abide to the new gymnasium-style API! [repeated 17x across cluster]
(RolloutWorker pid=252437) From Ray 2.3 on, RLlib only supports the new (gym>=0.26 or gymnasium) Env APIs. [repeated 17x across cluster]
(RolloutWorker pid=252437) In particular, the `reset()` method seems to be faulty. [repeated 17x across cluster]
(RolloutWorker pid=252437) Learn more about the most important changes here: [repeated 17x across cluster]
(RolloutWorker pid=252437) https://github.com/openai/gym and here: https://github.com/Farama-Foundation/Gymnasium [repeated 17x across cluster]
(RolloutWorker pid=252437) In order to fix this problem, do the following: [repeated 17x across cluster]
(RolloutWorker pid=252437) 1) Run `pip install gymnasium` on your command line. [repeated 17x across cluster]
(RolloutWorker pid=252437) 2) Change all your import statements in your code from [repeated 17x across cluster]
(RolloutWorker pid=252437) `import gym` -> `import gymnasium as gym` OR [repeated 17x across cluster]
(RolloutWorker pid=252437) `from gym.space import Discrete` -> `from gymnasium.spaces import Discrete` [repeated 17x across cluster]
(RolloutWorker pid=252437) For your custom (single agent) gym.Env classes: [repeated 17x across cluster]
(RolloutWorker pid=252437) 3.1) Either wrap your old Env class via the provided `from gymnasium.wrappers import [repeated 17x across cluster]
(RolloutWorker pid=252437) EnvCompatibility` wrapper class. [repeated 17x across cluster]
(RolloutWorker pid=252437) 4.2) Alternatively to 4.1: [repeated 34x across cluster]
(RolloutWorker pid=252437) - Change your `reset()` method to have the call signature 'def reset(self, *, [repeated 17x across cluster]
(RolloutWorker pid=252437) seed=None, options=None)' [repeated 17x across cluster]
(RolloutWorker pid=252437) - Return an additional info dict (empty dict should be fine) from your `reset()` [repeated 17x across cluster]
(RolloutWorker pid=252437) method. [repeated 17x across cluster]
(RolloutWorker pid=252437) - Return an additional `truncated` flag from your `step()` method (between `done` and [repeated 17x across cluster]
(RolloutWorker pid=252437) `info`). This flag should indicate, whether the episode was terminated prematurely [repeated 17x across cluster]
(RolloutWorker pid=252437) due to some time constraint or other kind of horizon setting. [repeated 17x across cluster]
(RolloutWorker pid=252437) For your custom RLlib `MultiAgentEnv` classes: [repeated 17x across cluster]
(RolloutWorker pid=252437) 4.1) Either wrap your old MultiAgentEnv via the provided [repeated 17x across cluster]
(RolloutWorker pid=252437) `from ray.rllib.env.wrappers.multi_agent_env_compatibility import [repeated 17x across cluster]
(RolloutWorker pid=252437) MultiAgentEnvCompatibility` wrapper class. [repeated 17x across cluster]
(RolloutWorker pid=252437) - Change your `reset()` method to have the call signature [repeated 17x across cluster]
(RolloutWorker pid=252437) 'def reset(self, *, seed=None, options=None)' [repeated 17x across cluster]
(RolloutWorker pid=252437) - Return an additional per-agent info dict (empty dict should be fine) from your [repeated 17x across cluster]
(RolloutWorker pid=252437) `reset()` method. [repeated 17x across cluster]
(RolloutWorker pid=252437) - Rename `dones` into `terminateds` and only set this to True, if the episode is really [repeated 17x across cluster]
(RolloutWorker pid=252437) done (as opposed to has been terminated prematurely due to some horizon/time-limit [repeated 17x across cluster]
(RolloutWorker pid=252437) setting). [repeated 17x across cluster]
(RolloutWorker pid=252437) - Return an additional `truncateds` per-agent dictionary flag from your `step()` [repeated 17x across cluster]
(RolloutWorker pid=252437) method, including the `__all__` key (100% analogous to your `dones/terminateds` [repeated 17x across cluster]
(RolloutWorker pid=252437) per-agent dict). [repeated 17x across cluster]
(RolloutWorker pid=252437) Return this new `truncateds` dict between `dones/terminateds` and `infos`. This [repeated 17x across cluster]
(RolloutWorker pid=252437) flag should indicate, whether the episode (for some agent or all agents) was [repeated 17x across cluster]
(RolloutWorker pid=252437) terminated prematurely due to some time constraint or other kind of horizon setting. [repeated 17x across cluster]
(RolloutWorker pid=252437) During handling of the above exception, another exception occurred: [repeated 9x across cluster]
(RolloutWorker pid=252437) File "/usr/local/lib/python3.10/dist-packages/ray/rllib/evaluation/rollout_worker.py", line 616, in __init__ [repeated 11x across cluster]
(RolloutWorker pid=252437) check_env(self.env) [repeated 8x across cluster]
(RolloutWorker pid=252437) ValueError: Traceback (most recent call last): [repeated 9x across cluster]
(RolloutWorker pid=252437) obs_and_infos = env.reset(seed=42, options={}) [repeated 9x across cluster]
(RolloutWorker pid=252437) The above error has been found in your environment! We've added a module for checking your custom environments. It may cause your experiment to fail if your environment is not set up correctly. You can disable this behavior via calling `config.environment(disable_env_checking=True)`. You can run the environment checking module standalone by calling ray.rllib.utils.check_env([your env]). [repeated 9x across cluster]Following is the versions of my libraries
(EE) root@3sec3sec3sec:/home/sam# pip list
Package Version
---------------------------- -------------
absl-py 1.4.0
aiosignal 1.3.1
apturl 0.5.2
astunparse 1.6.3
attrs 23.1.0
bcrypt 3.2.0
blinker 1.4
Brlapi 0.8.3
cachetools 5.3.1
certifi 2020.6.20
chardet 4.0.0
click 8.0.3
cloudpickle 2.2.1
cmake 3.26.4
colorama 0.4.4
command-not-found 0.3
cryptography 3.4.8
cupshelpers 1.0
dbus-python 1.2.18
decorator 5.1.1
defer 1.0.6
distro 1.7.0
distro-info 1.1build1
dm-tree 0.1.8
duplicity 0.8.21
Farama-Notifications 0.0.4
fasteners 0.14.1
filelock 3.12.2
flatbuffers 23.5.26
frozenlist 1.3.3
fsspec 2023.6.0
future 0.18.2
gast 0.4.0
google-auth 2.21.0
google-auth-oauthlib 1.0.0
google-pasta 0.2.0
grpcio 1.51.3
gymnasium 0.28.1
h5py 3.9.0
httplib2 0.20.2
idna 3.3
imageio 2.31.1
importlib-metadata 4.6.4
install 1.3.5
jax 0.4.13
jax-jumpy 1.0.0
jeepney 0.7.1
Jinja2 3.1.2
jsonschema 4.17.3
keras 2.12.0
keyring 23.5.0
language-selector 0.1
launchpadlib 1.10.16
lazr.restfulclient 0.14.4
lazr.uri 1.0.6
lazy_loader 0.3
libclang 16.0.0
lightdm-gtk-greeter-settings 1.2.2
lit 16.0.6
lockfile 0.12.2
louis 3.20.0
lz4 4.3.2
macaroonbakery 1.3.1
Mako 1.1.3
Markdown 3.4.3
MarkupSafe 2.1.3
menulibre 2.2.2
ml-dtypes 0.2.0
monotonic 1.6
more-itertools 8.10.0
mpmath 1.3.0
msgpack 1.0.5
mugshot 0.4.3
netifaces 0.11.0
networkx 3.1
numpy 1.23.5
nvidia-cublas-cu11 11.10.3.66
nvidia-cuda-cupti-cu11 11.7.101
nvidia-cuda-nvrtc-cu11 11.7.99
nvidia-cuda-runtime-cu11 11.7.99
nvidia-cudnn-cu11 8.5.0.96
nvidia-cufft-cu11 10.9.0.58
nvidia-curand-cu11 10.2.10.91
nvidia-cusolver-cu11 11.4.0.1
nvidia-cusparse-cu11 11.7.4.91
nvidia-nccl-cu11 2.14.3
nvidia-nvtx-cu11 11.7.91
oauthlib 3.2.0
olefile 0.46
onboard 1.4.1
opt-einsum 3.3.0
packaging 23.1
pandas 2.0.3
paramiko 2.9.3
pettingzoo 1.23.1
pexpect 4.8.0
Pillow 9.0.1
pip 23.1.2
pip-review 1.3.0
protobuf 4.23.3
psutil 5.9.0
ptyprocess 0.7.0
pyasn1 0.5.0
pyasn1-modules 0.3.0
pycairo 1.20.1
pycups 2.0.1
PyGObject 3.42.1
PyJWT 2.3.0
pymacaroons 0.13.0
PyNaCl 1.5.0
pyparsing 2.4.7
pyproj 3.3.0
pyRFC3339 1.1
pyrsistent 0.19.3
python-apt 2.4.0+ubuntu1
python-dateutil 2.8.2
python-debian 0.1.43ubuntu1
pytz 2022.1
PyWavelets 1.4.1
pyxdg 0.27
PyYAML 5.4.1
ray 2.5.1
reportlab 3.6.8
requests 2.25.1
requests-oauthlib 1.3.1
rsa 4.9
Rtree 0.9.7
scikit-image 0.21.0
scipy 1.11.1
SecretStorage 3.3.1
setuptools 59.6.0
six 1.16.0
sumo-rl 1.4.3
sumolib 1.18.0
svgwrite 1.4.3
sympy 1.12
systemd-python 234
tensorboard 2.12.3
tensorboard-data-server 0.7.1
tensorboardX 2.6.1
tensorflow 2.12.0
tensorflow-estimator 2.12.0
tensorflow-io-gcs-filesystem 0.32.0
tensorflow-probability 0.20.1
termcolor 2.3.0
tifffile 2023.4.12
torch 2.0.1
traci 1.18.0
triton 2.0.0
typing_extensions 4.7.0
tzdata 2023.3
ubuntu-advantage-tools 8001
ubuntu-drivers-common 0.0.0
ufw 0.36.1
unattended-upgrades 0.1
urllib3 1.26.5
usb-creator 0.3.7
wadllib 1.3.6
Werkzeug 2.3.6
wheel 0.37.1
wrapt 1.14.1
xdg 5
xkit 0.0.0
zipp 1.0.0Hope it is useful
Unamed318
commented
1 year ago I think this is similar to: 'https://github.com/ray-project/ray/issues/32889'
Unamed318
commented
1 year ago By replacing rllib/env/wrappers/pettingzoo_env.py to "https://github.com/ray-project/ray/blob/f24f94ca36a79366e935d842d657a25470216faa/rllib/env/wrappers/pettingzoo_env.py", it now works
sitexa
commented
1 year ago By replacing rllib/env/wrappers/pettingzoo_env.py to "https://github.com/ray-project/ray/blob/f24f94ca36a79366e935d842d657a25470216faa/rllib/env/wrappers/pettingzoo_env.py", it now works
It works.
Hello Im a student who has taken an interest in Machine Learning and tried to learn more about multi-agent env so I tried to run the example ppo_4x4grid.py but I couldnt seem to get it to work?
I would like help if possible in resolving it, thank you!