search

LLNL / Abmarl

Agent Based Modeling and Reinforcement Learning
Other
59 stars 17 forks source link

Superagent wrapper doesn't train with rllib 2.0.0 #397

Closed rusu24edward closed 1 year ago

rusu24edward commented 1 year ago

Team Battle Example works, but Team Battle Super Agent does not. Here's the error:

== Status ==
Current time: 2023-06-06 08:16:11 (running for 00:00:17.38)
Memory usage on this node: 76.0/125.5 GiB
Using FIFO scheduling algorithm.
Resources requested: 1.0/36 CPUs, 0/0 GPUs, 0.0/39.61 GiB heap, 0.0/19.8 GiB objects
Result logdir: /usr/WS1/rusu1/Abmarl/examples/output_dir/abmarl_results/TeamBattleSuperAgent_2023-06-06_08-15/A2C
Number of trials: 1/1 (1 RUNNING)

(A2C pid=2677849) 2023-06-06 08:16:11,368       ERROR worker.py:756 -- Exception raised in creation task: The actor died because of an error raised in its creation task, ray::A2C.__init__() (pid=2677849, ip=134.9.73.29, repr=A2C)
(A2C pid=2677849)   File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/algorithms/algorithm.py", line 308, in __init__
(A2C pid=2677849)     super().__init__(config=config, logger_creator=logger_creator, **kwargs)
(A2C pid=2677849)   File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/tune/trainable/trainable.py", line 157, in __init__
(A2C pid=2677849)     self.setup(copy.deepcopy(self.config))
(A2C pid=2677849)   File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/algorithms/a2c/a2c.py", line 139, in setup
(A2C pid=2677849)     super().setup(config)
(A2C pid=2677849)   File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/algorithms/a3c/a3c.py", line 161, in setup
(A2C pid=2677849)     super().setup(config)
(A2C pid=2677849)   File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/algorithms/algorithm.py", line 418, in setup
(A2C pid=2677849)     self.workers = WorkerSet(
(A2C pid=2677849)   File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/evaluation/worker_set.py", line 171, in __init__
(A2C pid=2677849)     self._local_worker = self._make_worker(
(A2C pid=2677849)   File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/evaluation/worker_set.py", line 661, in _make_worker
(A2C pid=2677849)     worker = cls(
(A2C pid=2677849)   File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/evaluation/rollout_worker.py", line 613, in __init__
(A2C pid=2677849)     self._build_policy_map(
(A2C pid=2677849)   File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/evaluation/rollout_worker.py", line 1784, in _build_policy_map
(A2C pid=2677849)     self.policy_map.create_policy(
(A2C pid=2677849)   File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/policy/policy_map.py", line 123, in create_policy
(A2C pid=2677849)     self[policy_id] = create_policy_for_framework(
(A2C pid=2677849)   File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/utils/policy.py", line 71, in create_policy_for_framework
(A2C pid=2677849)     return policy_class(
(A2C pid=2677849)   File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/algorithms/a3c/a3c_tf_policy.py", line 65, in __init__
(A2C pid=2677849)     base.__init__(
(A2C pid=2677849)   File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/policy/dynamic_tf_policy_v2.py", line 93, in __init__
(A2C pid=2677849)     ) = self._init_action_fetches(timestep, explore)
(A2C pid=2677849)   File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/policy/dynamic_tf_policy_v2.py", line 619, in _init_action_fetches
(A2C pid=2677849)     dist_inputs, self._state_out = self.model(self._input_dict)
(A2C pid=2677849)   File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/models/modelv2.py", line 247, in __call__
(A2C pid=2677849)     restored["obs"] = restore_original_dimensions(
(A2C pid=2677849)   File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/models/modelv2.py", line 411, in restore_original_dimensions
(A2C pid=2677849)     return _unpack_obs(obs, original_space, tensorlib=tensorlib)
(A2C pid=2677849)   File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/models/modelv2.py", line 445, in _unpack_obs
(A2C pid=2677849)     raise ValueError(
**(A2C pid=2677849) ValueError: Expected flattened obs shape of [..., 306], got (?, 300)**

2023-06-06 08:16:12,208 ERROR trial_runner.py:980 -- Trial A2C_TeamBattleSuperAgent_074f4_00000: Error processing event.
ray.tune.error._TuneNoNextExecutorEventError: Traceback (most recent call last):
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/tune/execution/ray_trial_executor.py", line 989, in get_next_executor_event
    future_result = ray.get(ready_future)
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/_private/client_mode_hook.py", line 105, in wrapper
    return func(*args, **kwargs)
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/_private/worker.py", line 2277, in get
    raise value
ray.exceptions.RayActorError: The actor died because of an error raised in its creation task, ray::A2C.__init__() (pid=2677849, ip=134.9.73.29, repr=A2C)
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/algorithms/algorithm.py", line 308, in __init__
    super().__init__(config=config, logger_creator=logger_creator, **kwargs)
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/tune/trainable/trainable.py", line 157, in __init__
    self.setup(copy.deepcopy(self.config))
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/algorithms/a2c/a2c.py", line 139, in setup
    super().setup(config)
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/algorithms/a3c/a3c.py", line 161, in setup
    super().setup(config)
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/algorithms/algorithm.py", line 418, in setup
    self.workers = WorkerSet(
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/evaluation/worker_set.py", line 171, in __init__
    self._local_worker = self._make_worker(
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/evaluation/worker_set.py", line 661, in _make_worker
    worker = cls(
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/evaluation/rollout_worker.py", line 613, in __init__
    self._build_policy_map(
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/evaluation/rollout_worker.py", line 1784, in _build_policy_map
    self.policy_map.create_policy(
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/policy/policy_map.py", line 123, in create_policy
    self[policy_id] = create_policy_for_framework(
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/utils/policy.py", line 71, in create_policy_for_framework
    return policy_class(
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/algorithms/a3c/a3c_tf_policy.py", line 65, in __init__
    base.__init__(
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/policy/dynamic_tf_policy_v2.py", line 93, in __init__
    ) = self._init_action_fetches(timestep, explore)
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/policy/dynamic_tf_policy_v2.py", line 619, in _init_action_fetches
    dist_inputs, self._state_out = self.model(self._input_dict)
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/models/modelv2.py", line 247, in __call__
    restored["obs"] = restore_original_dimensions(
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/models/modelv2.py", line 411, in restore_original_dimensions
    return _unpack_obs(obs, original_space, tensorlib=tensorlib)
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/models/modelv2.py", line 445, in _unpack_obs
    raise ValueError(
**ValueError: Expected flattened obs shape of [..., 306], got (?, 300)**

The trial A2C_TeamBattleSuperAgent_074f4_00000 errored with parameters={'disable_env_checking': False, 'env': 'TeamBattleSuperAgent', 'horizon': 200, 'env_config': {}, 'multiagent': {'policies': {'red': (None, Dict(agent0:Dict(grid:Box(-2, 4, (7, 7), int64)), agent12:Dict(grid:Box(-2, 4, (7, 7), int64)), agent16:Dict(grid:Box(-2, 4, (7, 7), int64)), agent20:Dict(grid:Box(-2, 4, (7, 7), int64)), agent4:Dict(grid:Box(-2, 4, (7, 7), int64)), agent8:Dict(grid:Box(-2, 4, (7, 7), int64)), mask:Dict(agent0:Discrete(2), agent12:Discrete(2), agent16:Discrete(2), agent20:Discrete(2), agent4:Discrete(2), agent8:Discrete(2))), Dict(agent0:Dict(attack:Discrete(2), move:Box(-1, 1, (2,), int64)), agent12:Dict(attack:Discrete(2), move:Box(-1, 1, (2,), int64)), agent16:Dict(attack:Discrete(2), move:Box(-1, 1, (2,), int64)), agent20:Dict(attack:Discrete(2), move:Box(-1, 1, (2,), int64)), agent4:Dict(attack:Discrete(2), move:Box(-1, 1, (2,), int64)), agent8:Dict(attack:Discrete(2), move:Box(-1, 1, (2,), int64))), {}), 'blue': (None, Dict(agent1:Dict(grid:Box(-2, 4, (7, 7), int64)), agent13:Dict(grid:Box(-2, 4, (7, 7), int64)), agent17:Dict(grid:Box(-2, 4, (7, 7), int64)), agent21:Dict(grid:Box(-2, 4, (7, 7), int64)), agent5:Dict(grid:Box(-2, 4, (7, 7), int64)), agent9:Dict(grid:Box(-2, 4, (7, 7), int64)), mask:Dict(agent1:Discrete(2), agent13:Discrete(2), agent17:Discrete(2), agent21:Discrete(2), agent5:Discrete(2), agent9:Discrete(2))), Dict(agent1:Dict(attack:Discrete(2), move:Box(-1, 1, (2,), int64)), agent13:Dict(attack:Discrete(2), move:Box(-1, 1, (2,), int64)), agent17:Dict(attack:Discrete(2), move:Box(-1, 1, (2,), int64)), agent21:Dict(attack:Discrete(2), move:Box(-1, 1, (2,), int64)), agent5:Dict(attack:Discrete(2), move:Box(-1, 1, (2,), int64)), agent9:Dict(attack:Discrete(2), move:Box(-1, 1, (2,), int64))), {}), 'green': (None, Dict(agent10:Dict(grid:Box(-2, 4, (7, 7), int64)), agent14:Dict(grid:Box(-2, 4, (7, 7), int64)), agent18:Dict(grid:Box(-2, 4, (7, 7), int64)), agent2:Dict(grid:Box(-2, 4, (7, 7), int64)), agent22:Dict(grid:Box(-2, 4, (7, 7), int64)), agent6:Dict(grid:Box(-2, 4, (7, 7), int64)), mask:Dict(agent10:Discrete(2), agent14:Discrete(2), agent18:Discrete(2), agent2:Discrete(2), agent22:Discrete(2), agent6:Discrete(2))), Dict(agent10:Dict(attack:Discrete(2), move:Box(-1, 1, (2,), int64)), agent14:Dict(attack:Discrete(2), move:Box(-1, 1, (2,), int64)), agent18:Dict(attack:Discrete(2), move:Box(-1, 1, (2,), int64)), agent2:Dict(attack:Discrete(2), move:Box(-1, 1, (2,), int64)), agent22:Dict(attack:Discrete(2), move:Box(-1, 1, (2,), int64)), agent6:Dict(attack:Discrete(2), move:Box(-1, 1, (2,), int64))), {}), 'gray': (None, Dict(agent11:Dict(grid:Box(-2, 4, (7, 7), int64)), agent15:Dict(grid:Box(-2, 4, (7, 7), int64)), agent19:Dict(grid:Box(-2, 4, (7, 7), int64)), agent23:Dict(grid:Box(-2, 4, (7, 7), int64)), agent3:Dict(grid:Box(-2, 4, (7, 7), int64)), agent7:Dict(grid:Box(-2, 4, (7, 7), int64)), mask:Dict(agent11:Discrete(2), agent15:Discrete(2), agent19:Discrete(2), agent23:Discrete(2), agent3:Discrete(2), agent7:Discrete(2))), Dict(agent11:Dict(attack:Discrete(2), move:Box(-1, 1, (2,), int64)), agent15:Dict(attack:Discrete(2), move:Box(-1, 1, (2,), int64)), agent19:Dict(attack:Discrete(2), move:Box(-1, 1, (2,), int64)), agent23:Dict(attack:Discrete(2), move:Box(-1, 1, (2,), int64)), agent3:Dict(attack:Discrete(2), move:Box(-1, 1, (2,), int64)), agent7:Dict(attack:Discrete(2), move:Box(-1, 1, (2,), int64))), {})}, 'policy_mapping_fn': <function policy_mapping_fn at 0x155546c0ed30>}, 'num_workers': 0, 'num_envs_per_worker': 1}. Error file: /usr/WS1/rusu1/Abmarl/examples/output_dir/abmarl_results/TeamBattleSuperAgent_2023-06-06_08-15/A2C/A2C_TeamBattleSuperAgent_074f4_00000_0_2023-06-06_08-15-54/error.txt
== Status ==
Current time: 2023-06-06 08:16:12 (running for 00:00:18.25)
Memory usage on this node: 76.0/125.5 GiB
Using FIFO scheduling algorithm.
Resources requested: 0/36 CPUs, 0/0 GPUs, 0.0/39.61 GiB heap, 0.0/19.8 GiB objects
Result logdir: /usr/WS1/rusu1/Abmarl/examples/output_dir/abmarl_results/TeamBattleSuperAgent_2023-06-06_08-15/A2C
Number of trials: 1/1 (1 ERROR)
Number of errored trials: 1
+--------------------------------------+--------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Trial name                           |   # failures | error file                                                                                                                                                              |
|--------------------------------------+--------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| A2C_TeamBattleSuperAgent_074f4_00000 |            1 | /usr/WS1/rusu1/Abmarl/examples/output_dir/abmarl_results/TeamBattleSuperAgent_2023-06-06_08-15/A2C/A2C_TeamBattleSuperAgent_074f4_00000_0_2023-06-06_08-15-54/error.txt |
+--------------------------------------+--------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

== Status ==
Current time: 2023-06-06 08:16:12 (running for 00:00:18.27)
Memory usage on this node: 76.0/125.5 GiB
Using FIFO scheduling algorithm.
Resources requested: 0/36 CPUs, 0/0 GPUs, 0.0/39.61 GiB heap, 0.0/19.8 GiB objects
Result logdir: /usr/WS1/rusu1/Abmarl/examples/output_dir/abmarl_results/TeamBattleSuperAgent_2023-06-06_08-15/A2C
Number of trials: 1/1 (1 ERROR)
+--------------------------------------+----------+-------+
| Trial name                           | status   | loc   |
|--------------------------------------+----------+-------|
| A2C_TeamBattleSuperAgent_074f4_00000 | ERROR    |       |
+--------------------------------------+----------+-------+
Number of errored trials: 1
+--------------------------------------+--------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Trial name                           |   # failures | error file                                                                                                                                                              |
|--------------------------------------+--------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| A2C_TeamBattleSuperAgent_074f4_00000 |            1 | /usr/WS1/rusu1/Abmarl/examples/output_dir/abmarl_results/TeamBattleSuperAgent_2023-06-06_08-15/A2C/A2C_TeamBattleSuperAgent_074f4_00000_0_2023-06-06_08-15-54/error.txt |
+--------------------------------------+--------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

2023-06-06 08:16:12,267 ERROR ray_trial_executor.py:103 -- An exception occurred when trying to stop the Ray actor:Traceback (most recent call last):
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/tune/execution/ray_trial_executor.py", line 94, in _post_stop_cleanup
    ray.get(future, timeout=0)
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/_private/client_mode_hook.py", line 105, in wrapper
    return func(*args, **kwargs)
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/_private/worker.py", line 2277, in get
    raise value
ray.exceptions.RayActorError: The actor died because of an error raised in its creation task, ray::A2C.__init__() (pid=2677849, ip=134.9.73.29, repr=A2C)
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/algorithms/algorithm.py", line 308, in __init__
    super().__init__(config=config, logger_creator=logger_creator, **kwargs)
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/tune/trainable/trainable.py", line 157, in __init__
    self.setup(copy.deepcopy(self.config))
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/algorithms/a2c/a2c.py", line 139, in setup
    super().setup(config)
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/algorithms/a3c/a3c.py", line 161, in setup
    super().setup(config)
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/algorithms/algorithm.py", line 418, in setup
    self.workers = WorkerSet(
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/evaluation/worker_set.py", line 171, in __init__
    self._local_worker = self._make_worker(
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/evaluation/worker_set.py", line 661, in _make_worker
    worker = cls(
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/evaluation/rollout_worker.py", line 613, in __init__
    self._build_policy_map(
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/evaluation/rollout_worker.py", line 1784, in _build_policy_map
    self.policy_map.create_policy(
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/policy/policy_map.py", line 123, in create_policy
    self[policy_id] = create_policy_for_framework(
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/utils/policy.py", line 71, in create_policy_for_framework
    return policy_class(
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/algorithms/a3c/a3c_tf_policy.py", line 65, in __init__
    base.__init__(
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/policy/dynamic_tf_policy_v2.py", line 93, in __init__
    ) = self._init_action_fetches(timestep, explore)
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/policy/dynamic_tf_policy_v2.py", line 619, in _init_action_fetches
    dist_inputs, self._state_out = self.model(self._input_dict)
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/models/modelv2.py", line 247, in __call__
    restored["obs"] = restore_original_dimensions(
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/models/modelv2.py", line 411, in restore_original_dimensions
    return _unpack_obs(obs, original_space, tensorlib=tensorlib)
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/models/modelv2.py", line 445, in _unpack_obs
    raise ValueError(
**ValueError: Expected flattened obs shape of [..., 306], got (?, 300)**

Traceback (most recent call last):
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/bin/abmarl", line 33, in <module>
    sys.exit(load_entry_point('abmarl', 'console_scripts', 'abmarl')())
  File "/usr/WS1/rusu1/Abmarl/abmarl/scripts/scripts.py", line 54, in cli
    train.run(path_config)
  File "/usr/WS1/rusu1/Abmarl/abmarl/scripts/train_script.py", line 17, in run
    train.run(full_config_path)
  File "/usr/WS1/rusu1/Abmarl/abmarl/train.py", line 30, in run
    tune.run(**experiment_mod.params['ray_tune'])
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/tune/tune.py", line 752, in run
    raise TuneError("Trials did not complete", incomplete_trials)
ray.tune.error.TuneError: ('Trials did not complete', [A2C_TeamBattleSuperAgent_074f4_00000])
(v_abmarl_39) [rusu1@pascal83:examples]$

Here's the content of the error file:

Failure # 1 (occurred at 2023-06-06_08-16-12)
Traceback (most recent call last):
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/tune/execution/ray_trial_executor.py", line 989, in get_next_executor_event
    future_result = ray.get(ready_future)
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/_private/client_mode_hook.py", line 105, in wrapper
    return func(*args, **kwargs)
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/_private/worker.py", line 2277, in get
    raise value
ray.exceptions.RayActorError: The actor died because of an error raised in its creation task, ray::A2C.__init__() (pid=2677849, ip=134.9.73.29, repr=A2C)
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/algorithms/algorithm.py", line 308, in __init__
    super().__init__(config=config, logger_creator=logger_creator, **kwargs)
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/tune/trainable/trainable.py", line 157, in __init__
    self.setup(copy.deepcopy(self.config))
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/algorithms/a2c/a2c.py", line 139, in setup
    super().setup(config)
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/algorithms/a3c/a3c.py", line 161, in setup
    super().setup(config)
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/algorithms/algorithm.py", line 418, in setup
    self.workers = WorkerSet(
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/evaluation/worker_set.py", line 171, in __init__
    self._local_worker = self._make_worker(
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/evaluation/worker_set.py", line 661, in _make_worker
    worker = cls(
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/evaluation/rollout_worker.py", line 613, in __init__
    self._build_policy_map(
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/evaluation/rollout_worker.py", line 1784, in _build_policy_map
    self.policy_map.create_policy(
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/policy/policy_map.py", line 123, in create_policy
    self[policy_id] = create_policy_for_framework(
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/utils/policy.py", line 71, in create_policy_for_framework
    return policy_class(
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/algorithms/a3c/a3c_tf_policy.py", line 65, in __init__
    base.__init__(
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/policy/dynamic_tf_policy_v2.py", line 93, in __init__
    ) = self._init_action_fetches(timestep, explore)
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/policy/dynamic_tf_policy_v2.py", line 619, in _init_action_fetches
    dist_inputs, self._state_out = self.model(self._input_dict)
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/models/modelv2.py", line 247, in __call__
    restored["obs"] = restore_original_dimensions(
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/models/modelv2.py", line 411, in restore_original_dimensions
    return _unpack_obs(obs, original_space, tensorlib=tensorlib)
  File "/usr/WS1/rusu1/virtual_envs/v_abmarl_39/lib/python3.9/site-packages/ray/rllib/models/modelv2.py", line 445, in _unpack_obs
    raise ValueError(
**ValueError: Expected flattened obs shape of [..., 306], got (?, 300)**
rusu24edward commented 1 year ago

Versions:

The following requires gymnasium instead of gym

rusu24edward commented 1 year ago

RLlib address this here: https://github.com/ray-project/ray/pull/31560. For the time being, I should modify the super agent wrapper so that it doesn't use Discrete space for the mask