Code runs a random amount (never more than 30 secs, then crashes)

[Jetpackjules]

WIth this error in conda:

 pid=12416) 2023-01-21 14:27:54,836     WARNING trainer.py:975 -- Worker crashed during call to `step_attempt()`. To try to continue training without the failed worker, set `ignore_worker_failures=True`.
Traceback (most recent call last):
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\Scripts\gdrl.exe\__main__.py", line 7, in <module>
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\godot_rl_agents\core\main.py", line 80, in main
    training_function(args)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\godot_rl_agents\wrappers\ray_wrapper.py", line 103, in rllib_training
    results = tune.run(
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\tune\tune.py", line 633, in run
    raise TuneError("Trials did not complete", incomplete_trials)
ray.tune.error.TuneError: ('Trials did not complete', [PPO_godot_cbdfc_00000])

[Jetpackjules]

Here is what I get when I add ignore_worker_failures=True to the ppo.yaml config:

(Sorry about how long it is the TLDR is that two array structures don't line up? (I think one or both is the _obs array...)

== Status ==
Current time: 2023-01-21 17:58:20 (running for 00:00:20.01)
Memory usage on this node: 12.1/32.0 GiB
Using FIFO scheduling algorithm.
Resources requested: 2.0/12 CPUs, 0/0 GPUs, 0.0/12.67 GiB heap, 0.0/6.34 GiB objects
Result logdir: C:\Users\Jetpackjules\ray_results\PPO/editor
Number of trials: 1/1 (1 RUNNING)
+-----------------------+----------+----------------+
| Trial name            | status   | loc            |
|-----------------------+----------+----------------|
| PPO_godot_33115_00000 | RUNNING  | 127.0.0.1:4628 |
+-----------------------+----------+----------------+

2023-01-21 17:58:24,060 ERROR trial_runner.py:920 -- Trial PPO_godot_33115_00000: Error processing event.
Traceback (most recent call last):
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\tune\trial_runner.py", line 886, in _process_trial
    results = self.trial_executor.fetch_result(trial)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\tune\ray_trial_executor.py", line 675, in fetch_result
    result = ray.get(trial_future[0], timeout=DEFAULT_GET_TIMEOUT)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\client_mode_hook.py", line 105, in wrapper
    return func(*args, **kwargs)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\worker.py", line 1763, in get
    raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(RuntimeError): ray::PPOTrainer.train() (pid=4628, ip=127.0.0.1, repr=PPOTrainer)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
    return method(self, *_args, **_kwargs)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 1044, in step_attempt
    step_results = self._exec_plan_or_training_iteration_fn()
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
    return method(self, *_args, **_kwargs)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 2032, in _exec_plan_or_training_iteration_fn
    results = next(self.train_exec_impl)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 756, in __next__
    return next(self.built_iterator)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
    for item in it:
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
    for item in it:
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 843, in apply_filter
    for item in it:
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 843, in apply_filter
    for item in it:
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
    for item in it:
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
    for item in it:
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
    for item in it:
  [Previous line repeated 1 more time]
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 876, in apply_flatten
    for item in it:
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
    for item in it:
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
    for item in it:
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
    for item in it:
  [Previous line repeated 1 more time]
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 471, in base_iterator
    yield ray.get(futures, timeout=timeout)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\client_mode_hook.py", line 105, in wrapper
    return func(*args, **kwargs)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\worker.py", line 1763, in get
    raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(ValueError): ray::RolloutWorker.par_iter_next() (pid=18520, ip=127.0.0.1, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x000001E38779DF10>)
ValueError: The two structures don't have the same nested structure.

First structure: type=dict str={'obs': [-3.29996368408203, -1.1629426574707, 0.982085049152374, 0.188438281416893, 0.480879679361979, 0.999991390700094, 0.9999898431357, 0.999984764703549, 0.999956677857643, 0, 0, 0, 0, 0, 0, 0, 0, 0.999956677857643, 0.999984764703549, 0.9999898431357, 0.999991390700094]}

Second structure: type=OrderedDict str=OrderedDict([('obs', array([-0.05282789, -0.07026779,  0.00310036,  0.2847447 ,  0.73455507,
        0.1685515 ,  0.73246104, -0.4114542 ,  0.42349654, -0.8441837 ,
       -0.998509  ,  0.08330388, -0.16747093,  0.9648661 , -0.07879055,
        0.2595761 ,  0.42683405, -0.45359635,  0.7723167 ,  0.6420327 ,
       -0.39561352], dtype=float32))])

More specifically: Substructure "type=list str=[-3.29996368408203, -1.1629426574707, 0.982085049152374, 0.188438281416893, 0.480879679361979, 0.999991390700094, 0.9999898431357, 0.999984764703549, 0.999956677857643, 0, 0, 0, 0, 0, 0, 0, 0, 0.999956677857643, 0.999984764703549, 0.9999898431357, 0.999991390700094]" is a sequence, while substructure "type=ndarray str=[-0.05282789 -0.07026779  0.00310036  0.2847447   0.73455507  0.1685515
  0.73246104 -0.4114542   0.42349654 -0.8441837  -0.998509    0.08330388
 -0.16747093  0.9648661  -0.07879055  0.2595761   0.42683405 -0.45359635
  0.7723167   0.6420327  -0.39561352]" is not

During handling of the above exception, another exception occurred:

ray::RolloutWorker.par_iter_next() (pid=18520, ip=127.0.0.1, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x000001E38779DF10>)
  File "python\ray\_raylet.pyx", line 633, in ray._raylet.execute_task
  File "python\ray\_raylet.pyx", line 674, in ray._raylet.execute_task
  File "python\ray\_raylet.pyx", line 640, in ray._raylet.execute_task
  File "python\ray\_raylet.pyx", line 644, in ray._raylet.execute_task
  File "python\ray\_raylet.pyx", line 593, in ray._raylet.execute_task.function_executor
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\function_manager.py", line 648, in actor_method_executor
    return method(__ray_actor, *args, **kwargs)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
    return method(self, *_args, **_kwargs)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 1151, in par_iter_next
    return next(self.local_it)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 380, in gen_rollouts
    yield self.sample()
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
    return method(self, *_args, **_kwargs)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 773, in sample
    batch = self.input_reader.next()
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 104, in next
    batches = [self.get_data()]
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 266, in get_data
    item = next(self._env_runner)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 634, in _env_runner
    _process_observations(
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 852, in _process_observations
    prep_obs = preprocessor.transform(raw_obs)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\models\preprocessors.py", line 275, in transform
    self.check_shape(observation)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\models\preprocessors.py", line 68, in check_shape
    observation = convert_element_to_space_type(
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\utils\spaces\space_utils.py", line 321, in convert_element_to_space_type
    return tree.map_structure(
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\tree\__init__.py", line 507, in map_structure
    assert_same_structure(structures[0], other, check_types=check_types)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\tree\__init__.py", line 363, in assert_same_structure
    raise type(e)("%s\n"
ValueError: The two structures don't have the same nested structure.

First structure: type=dict str={'obs': [-3.29996368408203, -1.1629426574707, 0.982085049152374, 0.188438281416893, 0.480879679361979, 0.999991390700094, 0.9999898431357, 0.999984764703549, 0.999956677857643, 0, 0, 0, 0, 0, 0, 0, 0, 0.999956677857643, 0.999984764703549, 0.9999898431357, 0.999991390700094]}

Second structure: type=OrderedDict str=OrderedDict([('obs', array([-0.05282789, -0.07026779,  0.00310036,  0.2847447 ,  0.73455507,
        0.1685515 ,  0.73246104, -0.4114542 ,  0.42349654, -0.8441837 ,
       -0.998509  ,  0.08330388, -0.16747093,  0.9648661 , -0.07879055,
        0.2595761 ,  0.42683405, -0.45359635,  0.7723167 ,  0.6420327 ,
       -0.39561352], dtype=float32))])

More specifically: Substructure "type=list str=[-3.29996368408203, -1.1629426574707, 0.982085049152374, 0.188438281416893, 0.480879679361979, 0.999991390700094, 0.9999898431357, 0.999984764703549, 0.999956677857643, 0, 0, 0, 0, 0, 0, 0, 0, 0.999956677857643, 0.999984764703549, 0.9999898431357, 0.999991390700094]" is a sequence, while substructure "type=ndarray str=[-0.05282789 -0.07026779  0.00310036  0.2847447   0.73455507  0.1685515
  0.73246104 -0.4114542   0.42349654 -0.8441837  -0.998509    0.08330388
 -0.16747093  0.9648661  -0.07879055  0.2595761   0.42683405 -0.45359635
  0.7723167   0.6420327  -0.39561352]" is not
Entire first structure:
{'obs': [., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., .]}
Entire second structure:
OrderedDict([('obs', .)])

During handling of the above exception, another exception occurred:

ray::PPOTrainer.train() (pid=4628, ip=127.0.0.1, repr=PPOTrainer)
  File "python\ray\_raylet.pyx", line 633, in ray._raylet.execute_task
  File "python\ray\_raylet.pyx", line 674, in ray._raylet.execute_task
  File "python\ray\_raylet.pyx", line 640, in ray._raylet.execute_task
  File "python\ray\_raylet.pyx", line 644, in ray._raylet.execute_task
  File "python\ray\_raylet.pyx", line 593, in ray._raylet.execute_task.function_executor
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\function_manager.py", line 648, in actor_method_executor
    return method(__ray_actor, *args, **kwargs)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
    return method(self, *_args, **_kwargs)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\tune\trainable.py", line 319, in train
    result = self.step()
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
    return method(self, *_args, **_kwargs)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 972, in step
    self.try_recover_from_step_attempt()
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
    return method(self, *_args, **_kwargs)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 2434, in try_recover_from_step_attempt
    raise RuntimeError(
RuntimeError: Not enough healthy workers remain to continue.
Result for PPO_godot_33115_00000:
  date: 2023-01-21_17-58-09
  experiment_id: 2167004edda749d38be4874040f2eb86
  hostname: DESKTOP-J8BR5AT
  node_ip: 127.0.0.1
  pid: 4628
  timestamp: 1674352689
  trial_id: '33115_00000'

== Status ==
Current time: 2023-01-21 17:58:24 (running for 00:00:23.45)
Memory usage on this node: 12.1/32.0 GiB
Using FIFO scheduling algorithm.
Resources requested: 0/12 CPUs, 0/0 GPUs, 0.0/12.67 GiB heap, 0.0/6.34 GiB objects
Result logdir: C:\Users\Jetpackjules\ray_results\PPO/editor
Number of trials: 1/1 (1 ERROR)
+-----------------------+----------+----------------+
| Trial name            | status   | loc            |
|-----------------------+----------+----------------|
| PPO_godot_33115_00000 | ERROR    | 127.0.0.1:4628 |
+-----------------------+----------+----------------+
Number of errored trials: 1
+-----------------------+--------------+----------------------------------------------------------------------------------------------------+
| Trial name            |   # failures | error file                                                                                         |
|-----------------------+--------------+----------------------------------------------------------------------------------------------------|
| PPO_godot_33115_00000 |            1 | C:\Users\Jetpackjules\ray_results\PPO/editor\PPO_godot_33115_00000_0_2023-01-21_17-58-00\error.txt |
+-----------------------+--------------+----------------------------------------------------------------------------------------------------+

 pid=4628) 2023-01-21 17:58:24,052      ERROR trainer.py:970 -- Error in train call, attempting to recover
 pid=4628) Traceback (most recent call last):
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 965, in step
 pid=4628)     step_attempt_results = self.step_attempt()
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
 pid=4628)     return method(self, *_args, **_kwargs)
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 1044, in step_attempt
 pid=4628)     step_results = self._exec_plan_or_training_iteration_fn()
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
 pid=4628)     return method(self, *_args, **_kwargs)
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 2032, in _exec_plan_or_training_iteration_fn
 pid=4628)     results = next(self.train_exec_impl)
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 756, in __next__
 pid=4628)     return next(self.built_iterator)
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=4628)     for item in it:
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=4628)     for item in it:
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 843, in apply_filter
 pid=4628)     for item in it:
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 843, in apply_filter
 pid=4628)     for item in it:
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=4628)     for item in it:
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=4628)     for item in it:
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=4628)     for item in it:
 pid=4628)   [Previous line repeated 1 more time]
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 876, in apply_flatten
 pid=4628)     for item in it:
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=4628)     for item in it:
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=4628)     for item in it:
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=4628)     for item in it:
 pid=4628)   [Previous line repeated 1 more time]
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 471, in base_iterator
 pid=4628)     yield ray.get(futures, timeout=timeout)
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\client_mode_hook.py", line 105, in wrapper
 pid=4628)     return func(*args, **kwargs)
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\worker.py", line 1763, in get
 pid=4628)     raise value.as_instanceof_cause()
 pid=4628) ray.exceptions.RayTaskError(ValueError): ray::RolloutWorker.par_iter_next() (pid=18520, ip=127.0.0.1, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x000001E38779DF10>)
 pid=4628) ValueError: The two structures don't have the same nested structure.
 pid=4628)
 pid=4628) First structure: type=dict str={'obs': [-3.29996368408203, -1.1629426574707, 0.982085049152374, 0.188438281416893, 0.480879679361979, 0.999991390700094, 0.9999898431357, 0.999984764703549, 0.999956677857643, 0, 0, 0, 0, 0, 0, 0, 0, 0.999956677857643, 0.999984764703549, 0.9999898431357, 0.999991390700094]}
 pid=4628)
 pid=4628) Second structure: type=OrderedDict str=OrderedDict([('obs', array([-0.05282789, -0.07026779,  0.00310036,  0.2847447 ,  0.73455507,
 pid=4628)         0.1685515 ,  0.73246104, -0.4114542 ,  0.42349654, -0.8441837 ,
 pid=4628)        -0.998509  ,  0.08330388, -0.16747093,  0.9648661 , -0.07879055,
 pid=4628)         0.2595761 ,  0.42683405, -0.45359635,  0.7723167 ,  0.6420327 ,
 pid=4628)        -0.39561352], dtype=float32))])
 pid=4628)
 pid=4628) More specifically: Substructure "type=list str=[-3.29996368408203, -1.1629426574707, 0.982085049152374, 0.188438281416893, 0.480879679361979, 0.999991390700094, 0.9999898431357, 0.999984764703549, 0.999956677857643, 0, 0, 0, 0, 0, 0, 0, 0, 0.999956677857643, 0.999984764703549, 0.9999898431357, 0.999991390700094]" is a sequence, while substructure "type=ndarray str=[-0.05282789 -0.07026779  0.00310036  0.2847447   0.73455507  0.1685515
 pid=4628)   0.73246104 -0.4114542   0.42349654 -0.8441837  -0.998509    0.08330388
 pid=4628)  -0.16747093  0.9648661  -0.07879055  0.2595761   0.42683405 -0.45359635
 pid=4628)   0.7723167   0.6420327  -0.39561352]" is not
 pid=4628)
 pid=4628) During handling of the above exception, another exception occurred:
 pid=4628)
 pid=4628) ray::RolloutWorker.par_iter_next() (pid=18520, ip=127.0.0.1, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x000001E38779DF10>)
 pid=4628)   File "python\ray\_raylet.pyx", line 633, in ray._raylet.execute_task
 pid=4628)   File "python\ray\_raylet.pyx", line 674, in ray._raylet.execute_task
 pid=4628)   File "python\ray\_raylet.pyx", line 640, in ray._raylet.execute_task
 pid=4628)   File "python\ray\_raylet.pyx", line 644, in ray._raylet.execute_task
 pid=4628)   File "python\ray\_raylet.pyx", line 593, in ray._raylet.execute_task.function_executor
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\function_manager.py", line 648, in actor_method_executor
 pid=4628)     return method(__ray_actor, *args, **kwargs)
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
 pid=4628)     return method(self, *_args, **_kwargs)
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 1151, in par_iter_next
 pid=4628)     return next(self.local_it)
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 380, in gen_rollouts
 pid=4628)     yield self.sample()
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
 pid=4628)     return method(self, *_args, **_kwargs)
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 773, in sample
 pid=4628)     batch = self.input_reader.next()
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 104, in next
 pid=4628)     batches = [self.get_data()]
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 266, in get_data
 pid=4628)     item = next(self._env_runner)
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 634, in _env_runner
 pid=4628)     _process_observations(
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 852, in _process_observations
 pid=4628)     prep_obs = preprocessor.transform(raw_obs)
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\models\preprocessors.py", line 275, in transform
 pid=4628)     self.check_shape(observation)
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\models\preprocessors.py", line 68, in check_shape
 pid=4628)     observation = convert_element_to_space_type(
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\utils\spaces\space_utils.py", line 321, in convert_element_to_space_type
 pid=4628)     return tree.map_structure(
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\tree\__init__.py", line 507, in map_structure
 pid=4628)     assert_same_structure(structures[0], other, check_types=check_types)
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\tree\__init__.py", line 363, in assert_same_structure
 pid=4628)     raise type(e)("%s\n"
 pid=4628) ValueError: The two structures don't have the same nested structure.
 pid=4628)
 pid=4628) First structure: type=dict str={'obs': [-3.29996368408203, -1.1629426574707, 0.982085049152374, 0.188438281416893, 0.480879679361979, 0.999991390700094, 0.9999898431357, 0.999984764703549, 0.999956677857643, 0, 0, 0, 0, 0, 0, 0, 0, 0.999956677857643, 0.999984764703549, 0.9999898431357, 0.999991390700094]}
 pid=4628)
 pid=4628) Second structure: type=OrderedDict str=OrderedDict([('obs', array([-0.05282789, -0.07026779,  0.00310036,  0.2847447 ,  0.73455507,
 pid=4628)         0.1685515 ,  0.73246104, -0.4114542 ,  0.42349654, -0.8441837 ,
 pid=4628)        -0.998509  ,  0.08330388, -0.16747093,  0.9648661 , -0.07879055,
 pid=4628)         0.2595761 ,  0.42683405, -0.45359635,  0.7723167 ,  0.6420327 ,
 pid=4628)        -0.39561352], dtype=float32))])
 pid=4628)
 pid=4628) More specifically: Substructure "type=list str=[-3.29996368408203, -1.1629426574707, 0.982085049152374, 0.188438281416893, 0.480879679361979, 0.999991390700094, 0.9999898431357, 0.999984764703549, 0.999956677857643, 0, 0, 0, 0, 0, 0, 0, 0, 0.999956677857643, 0.999984764703549, 0.9999898431357, 0.999991390700094]" is a sequence, while substructure "type=ndarray str=[-0.05282789 -0.07026779  0.00310036  0.2847447   0.73455507  0.1685515
 pid=4628)   0.73246104 -0.4114542   0.42349654 -0.8441837  -0.998509    0.08330388
 pid=4628)  -0.16747093  0.9648661  -0.07879055  0.2595761   0.42683405 -0.45359635
 pid=4628)   0.7723167   0.6420327  -0.39561352]" is not
 pid=4628) Entire first structure:
 pid=4628) {'obs': [., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., .]}
 pid=4628) Entire second structure:
 pid=4628) OrderedDict([('obs', .)])
 pid=4628) 2023-01-21 17:58:24,056      ERROR trainer.py:2427 -- Removing unhealthy worker 1
 pid=4628) Traceback (most recent call last):
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 965, in step
 pid=4628)     step_attempt_results = self.step_attempt()
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
 pid=4628)     return method(self, *_args, **_kwargs)
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 1044, in step_attempt
 pid=4628)     step_results = self._exec_plan_or_training_iteration_fn()
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
 pid=4628)     return method(self, *_args, **_kwargs)
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 2032, in _exec_plan_or_training_iteration_fn
 pid=4628)     results = next(self.train_exec_impl)
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 756, in __next__
 pid=4628)     return next(self.built_iterator)
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=4628)     for item in it:
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=4628)     for item in it:
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 843, in apply_filter
 pid=4628)     for item in it:
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 843, in apply_filter
 pid=4628)     for item in it:
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=4628)     for item in it:
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=4628)     for item in it:
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=4628)     for item in it:
 pid=4628)   [Previous line repeated 1 more time]
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 876, in apply_flatten
 pid=4628)     for item in it:
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=4628)     for item in it:
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=4628)     for item in it:
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=4628)     for item in it:
 pid=4628)   [Previous line repeated 1 more time]
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 471, in base_iterator
 pid=4628)     yield ray.get(futures, timeout=timeout)
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\client_mode_hook.py", line 105, in wrapper
 pid=4628)     return func(*args, **kwargs)
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\worker.py", line 1763, in get
 pid=4628)     raise value.as_instanceof_cause()
 pid=4628) ray.exceptions.RayTaskError(ValueError): ray::RolloutWorker.par_iter_next() (pid=18520, ip=127.0.0.1, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x000001E38779DF10>)
 pid=4628) ValueError: The two structures don't have the same nested structure.
 pid=4628)
 pid=4628) First structure: type=dict str={'obs': [-3.29996368408203, -1.1629426574707, 0.982085049152374, 0.188438281416893, 0.480879679361979, 0.999991390700094, 0.9999898431357, 0.999984764703549, 0.999956677857643, 0, 0, 0, 0, 0, 0, 0, 0, 0.999956677857643, 0.999984764703549, 0.9999898431357, 0.999991390700094]}
 pid=4628)
 pid=4628) Second structure: type=OrderedDict str=OrderedDict([('obs', array([-0.05282789, -0.07026779,  0.00310036,  0.2847447 ,  0.73455507,
 pid=4628)         0.1685515 ,  0.73246104, -0.4114542 ,  0.42349654, -0.8441837 ,
 pid=4628)        -0.998509  ,  0.08330388, -0.16747093,  0.9648661 , -0.07879055,
 pid=4628)         0.2595761 ,  0.42683405, -0.45359635,  0.7723167 ,  0.6420327 ,
 pid=4628)        -0.39561352], dtype=float32))])
 pid=4628)
 pid=4628) More specifically: Substructure "type=list str=[-3.29996368408203, -1.1629426574707, 0.982085049152374, 0.188438281416893, 0.480879679361979, 0.999991390700094, 0.9999898431357, 0.999984764703549, 0.999956677857643, 0, 0, 0, 0, 0, 0, 0, 0, 0.999956677857643, 0.999984764703549, 0.9999898431357, 0.999991390700094]" is a sequence, while substructure "type=ndarray str=[-0.05282789 -0.07026779  0.00310036  0.2847447   0.73455507  0.1685515
 pid=4628)   0.73246104 -0.4114542   0.42349654 -0.8441837  -0.998509    0.08330388
 pid=4628)  -0.16747093  0.9648661  -0.07879055  0.2595761   0.42683405 -0.45359635
 pid=4628)   0.7723167   0.6420327  -0.39561352]" is not
 pid=4628)
 pid=4628) During handling of the above exception, another exception occurred:
 pid=4628)
 pid=4628) ray::RolloutWorker.par_iter_next() (pid=18520, ip=127.0.0.1, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x000001E38779DF10>)
 pid=4628)   File "python\ray\_raylet.pyx", line 633, in ray._raylet.execute_task
 pid=4628)   File "python\ray\_raylet.pyx", line 674, in ray._raylet.execute_task
 pid=4628)   File "python\ray\_raylet.pyx", line 640, in ray._raylet.execute_task
 pid=4628)   File "python\ray\_raylet.pyx", line 644, in ray._raylet.execute_task
 pid=4628)   File "python\ray\_raylet.pyx", line 593, in ray._raylet.execute_task.function_executor
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\function_manager.py", line 648, in actor_method_executor
 pid=4628)     return method(__ray_actor, *args, **kwargs)
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
 pid=4628)     return method(self, *_args, **_kwargs)
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 1151, in par_iter_next
 pid=4628)     return next(self.local_it)
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 380, in gen_rollouts
 pid=4628)     yield self.sample()
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
 pid=4628)     return method(self, *_args, **_kwargs)
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 773, in sample
 pid=4628)     batch = self.input_reader.next()
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 104, in next
 pid=4628)     batches = [self.get_data()]
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 266, in get_data
 pid=4628)     item = next(self._env_runner)
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 634, in _env_runner
 pid=4628)     _process_observations(
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 852, in _process_observations
 pid=4628)     prep_obs = preprocessor.transform(raw_obs)
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\models\preprocessors.py", line 275, in transform
 pid=4628)     self.check_shape(observation)
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\models\preprocessors.py", line 68, in check_shape
 pid=4628)     observation = convert_element_to_space_type(
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\utils\spaces\space_utils.py", line 321, in convert_element_to_space_type
 pid=4628)     return tree.map_structure(
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\tree\__init__.py", line 507, in map_structure
 pid=4628)     assert_same_structure(structures[0], other, check_types=check_types)
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\tree\__init__.py", line 363, in assert_same_structure
 pid=4628)     raise type(e)("%s\n"
 pid=4628) ValueError: The two structures don't have the same nested structure.
 pid=4628)
 pid=4628) First structure: type=dict str={'obs': [-3.29996368408203, -1.1629426574707, 0.982085049152374, 0.188438281416893, 0.480879679361979, 0.999991390700094, 0.9999898431357, 0.999984764703549, 0.999956677857643, 0, 0, 0, 0, 0, 0, 0, 0, 0.999956677857643, 0.999984764703549, 0.9999898431357, 0.999991390700094]}
 pid=4628)
 pid=4628) Second structure: type=OrderedDict str=OrderedDict([('obs', array([-0.05282789, -0.07026779,  0.00310036,  0.2847447 ,  0.73455507,
 pid=4628)         0.1685515 ,  0.73246104, -0.4114542 ,  0.42349654, -0.8441837 ,
 pid=4628)        -0.998509  ,  0.08330388, -0.16747093,  0.9648661 , -0.07879055,
 pid=4628)         0.2595761 ,  0.42683405, -0.45359635,  0.7723167 ,  0.6420327 ,
 pid=4628)        -0.39561352], dtype=float32))])
 pid=4628)
 pid=4628) More specifically: Substructure "type=list str=[-3.29996368408203, -1.1629426574707, 0.982085049152374, 0.188438281416893, 0.480879679361979, 0.999991390700094, 0.9999898431357, 0.999984764703549, 0.999956677857643, 0, 0, 0, 0, 0, 0, 0, 0, 0.999956677857643, 0.999984764703549, 0.9999898431357, 0.999991390700094]" is a sequence, while substructure "type=ndarray str=[-0.05282789 -0.07026779  0.00310036  0.2847447   0.73455507  0.1685515
 pid=4628)   0.73246104 -0.4114542   0.42349654 -0.8441837  -0.998509    0.08330388
 pid=4628)  -0.16747093  0.9648661  -0.07879055  0.2595761   0.42683405 -0.45359635
 pid=4628)   0.7723167   0.6420327  -0.39561352]" is not
 pid=4628) Entire first structure:
 pid=4628) {'obs': [., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., .]}
 pid=4628) Entire second structure:
 pid=4628) OrderedDict([('obs', .)])
 pid=4628)
 pid=4628) During handling of the above exception, another exception occurred:
 pid=4628)
 pid=4628) Traceback (most recent call last):
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 2423, in try_recover_from_step_attempt
 pid=4628)     ray.get(obj_ref)
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\client_mode_hook.py", line 105, in wrapper
 pid=4628)     return func(*args, **kwargs)
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\worker.py", line 1763, in get
 pid=4628)     raise value.as_instanceof_cause()
 pid=4628) ray.exceptions.RayTaskError(StopIteration): ray::RolloutWorker.sample_with_count() (pid=18520, ip=127.0.0.1, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x000001E38779DF10>)
 pid=4628)   File "python\ray\_raylet.pyx", line 640, in ray._raylet.execute_task
 pid=4628)   File "python\ray\_raylet.pyx", line 644, in ray._raylet.execute_task
 pid=4628)   File "python\ray\_raylet.pyx", line 593, in ray._raylet.execute_task.function_executor
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\function_manager.py", line 648, in actor_method_executor
 pid=4628)     return method(__ray_actor, *args, **kwargs)
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
 pid=4628)     return method(self, *_args, **_kwargs)
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 817, in sample_with_count
 pid=4628)     batch = self.sample()
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
 pid=4628)     return method(self, *_args, **_kwargs)
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 759, in sample
 pid=4628)     batches = [self.input_reader.next()]
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 104, in next
 pid=4628)     batches = [self.get_data()]
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 266, in get_data
 pid=4628)     item = next(self._env_runner)
 pid=4628) StopIteration
 pid=4628) 2023-01-21 17:58:24,059      ERROR worker.py:83 -- Unhandled error (suppress with RAY_IGNORE_UNHANDLED_ERRORS=1): ray::RolloutWorker.sample_with_count() (pid=18520, ip=127.0.0.1, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x000001E38779DF10>)
 pid=4628)   File "python\ray\_raylet.pyx", line 640, in ray._raylet.execute_task
 pid=4628)   File "python\ray\_raylet.pyx", line 644, in ray._raylet.execute_task
 pid=4628)   File "python\ray\_raylet.pyx", line 593, in ray._raylet.execute_task.function_executor
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\function_manager.py", line 648, in actor_method_executor
 pid=4628)     return method(__ray_actor, *args, **kwargs)
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
 pid=4628)     return method(self, *_args, **_kwargs)
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 817, in sample_with_count
 pid=4628)     batch = self.sample()
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
 pid=4628)     return method(self, *_args, **_kwargs)
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 759, in sample
 pid=4628)     batches = [self.input_reader.next()]
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 104, in next
 pid=4628)     batches = [self.get_data()]
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 266, in get_data
 pid=4628)     item = next(self._env_runner)
 pid=4628) StopIteration
 pid=4628) 2023-01-21 17:58:24,516      ERROR worker_set.py:219 -- Failed to stop workers!
 pid=4628) Traceback (most recent call last):
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\worker_set.py", line 217, in stop
 pid=4628)     ray.get(tids)
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\client_mode_hook.py", line 105, in wrapper
 pid=4628)     return func(*args, **kwargs)
 pid=4628)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\worker.py", line 1765, in get
 pid=4628)     raise value
 pid=4628) ray.exceptions.RayActorError: The actor died unexpectedly before finishing this task.
 pid=4628)      class_name: RolloutWorker
 pid=4628)      actor_id: acff4d96eb6b2e8414ec23cd01000000
 pid=4628)      pid: 18520
 pid=4628)      namespace: c05e4d0b-43ea-4338-a4b0-82c0d4377df9
 pid=4628)      ip: 127.0.0.1
 pid=4628) The actor is dead because its worker process has died. Worker exit type: INTENDED_EXIT
Traceback (most recent call last):
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\Scripts\gdrl.exe\__main__.py", line 7, in <module>
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\godot_rl_agents\core\main.py", line 80, in main
    training_function(args)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\godot_rl_agents\wrappers\ray_wrapper.py", line 103, in rllib_training
    results = tune.run(
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\tune\tune.py", line 633, in run
    raise TuneError("Trials did not complete", incomplete_trials)
ray.tune.error.TuneError: ('Trials did not complete', [PPO_godot_33115_00000])
 pid=18520) exit was not clean, using atexit to close env
 pid=18520) close message sent

[Jetpackjules]

I've tried a bunch of stuff (ensuring there is a reward, reducing ray casts, completely removing ray casts, etc.), to no avail.

Could this have something to do with raylib or sb3? I am just running gdrl in the conda terminal in my project path. Which 3 models is this calling?

[edbeeching]

Are you sure your observations are always the same length? If you replace the obs with a dummy value, like [0.0,0.0,0.0,0.0,0.0], does the error still occur?

[Jetpackjules]

Thank you for the suggeston.

I replaced the get_obs() function with this:

func get_obs():
    var relative = leaf.position - position
    var distance = relative.length() / 1500.0
    relative = relative.normalized()
    var result := []
    result.append(((position.x / WIDTH)-0.5) * 2)
    result.append(((position.y / HEIGHT)-0.5) * 2)
    result.append(relative.x)
    result.append(relative.y)
    result.append(distance)
    var raycast_obs = raycast_sensor.get_observation()
    result.append_array(raycast_obs)
    print("OBS OUT:")
    result = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
    print(result)

    return {
        "obs": result
    }

And the error never appeared!

Hope this helps.

[edbeeching]

Ok, we are getting somewhere. So either it is the raycast sensor, or the other part of the obs. You mentioned that you have tried without the raycast sensor obs being appended to the array? Have you tried with just the raycast sensor? It is not clear to me what would have caused the error.

[Jetpackjules]

Can you elaborate? I don't think I understand you the request. Do you mean without adding the raycast sensor data to the ‘results’ array?

Get Outlook for iOShttps://aka.ms/o0ukef

---

From: Edward Beeching @.> Sent: Saturday, January 28, 2023 11:19:13 AM To: edbeeching/godot_rl_agents @.> Cc: Jetpackjules @.>; Author @.> Subject: Re: [edbeeching/godot_rl_agents] Code runs a random amount (never more than 30 secs, then crashes) (Issue #53)

Ok, we are getting somewhere. So either it is the raycast sensor, or the other part of the obs. You mentioned that you have tried without the raycast sensor obs being appended to the array? Have you tried with just the raycast sensor? It is not clear to me what would have caused the error.

— Reply to this email directly, view it on GitHubhttps://github.com/edbeeching/godot_rl_agents/issues/53#issuecomment-1407467861, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ALGCRH6T7MCWLB533QRF4DDWUVWLDANCNFSM6AAAAAAUCUMLBU. You are receiving this because you authored the thread.Message ID: @.***>

[edbeeching]

I would run two things:

1. Having the result array being just the raycast sensor data
2. Having the result array be everything else except the raycast sensor data

That should help narrow does where the issue is coming from.

[Jetpackjules]

gotcha. Will try that now.

[Jetpackjules]

Ok, so this is weird.

The program never crashed when I removed the raycast data or removed everything else, but still does with both...

[Jetpackjules]

I fixed it! (I think).

I was only using one scene, but when I took the steps to add more (16) so they could simultaneously train, the program did not crash, and my agents visibly learn!

Was something from the ppo.yaml causing issues? Not sure why this works now, but it does...

[edbeeching]

Ah perhaps you did not update the num_envs_per_worker=16 to 1? Closing as you solved it.

[Jetpackjules]

Unfortunately, It's happening again :(. It sometimes works fine for like an hour, and sometimes I can't for the life of me get it to go more than 5 seconds....

Here is the error I get:

(gdrl_conda) PS C:\Users\Jetpackjules\Documents\Godot Gamez\ANTv0.61> gdrl
run_name PPO/editor
SETTING WORKERS TO 1
{'algorithm': 'PPO', 'RAY_IGNORE_UNHANDLED_ERRORS': 1, 'reuse_actors': True, 'stop': {'episode_reward_mean': 50000, 'training_iteration': 1000000, 'timesteps_total': 2000000000}, 'config': {'env': 'godot', 'env_config': {'framerate': None, 'action_repeat': None, 'show_window': True, 'seed': 0, 'env_path': None}, 'framework': 'torch', 'lambda': 0.95, 'gamma': 0.95, 'ignore_worker_failures': True, 'vf_clip_param': 200.0, 'clip_param': 0.2, 'entropy_coeff': 0.001, 'entropy_coeff_schedule': None, 'train_batch_size': 1024, 'sgd_minibatch_size': 128, 'num_sgd_iter': 16, 'num_workers': 1, 'lr': 0.0003, 'num_envs_per_worker': 1, 'batch_mode': 'truncate_episodes', 'rollout_fragment_length': 16, 'num_gpus': 0, 'model': {'fcnet_hiddens': [256, 256], 'use_lstm': False, 'lstm_cell_size': 32, 'framestack': 4}, 'no_done_at_end': False, 'soft_horizon': False}}
2023-02-06 15:31:28,321 ERROR syncer.py:111 -- Log sync requires rsync to be installed.
 pid=3084) 2023-02-06 15:31:31,471      INFO ppo.py:249 -- In multi-agent mode, policies will be optimized sequentially by the multi-GPU optimizer. Consider setting simple_optimizer=True if this doesn't work for you.
 pid=3084) 2023-02-06 15:31:31,471      INFO trainer.py:779 -- Current log_level is WARN. For more information, set 'log_level': 'INFO' / 'DEBUG' or use the -v and -vv flags.
 pid=13392) No game binary has been provided, please press PLAY in the Godot editor
 pid=13392) waiting for remote GODOT connection on port 11008
== Status ==
Current time: 2023-02-06 15:32:19 (running for 00:00:51.37)
Memory usage on this node: 13.9/32.0 GiB
Using FIFO scheduling algorithm.
Resources requested: 2.0/12 CPUs, 0/0 GPUs, 0.0/11.75 GiB heap, 0.0/5.87 GiB objects
Result logdir: C:\Users\Jetpackjules\ray_results\PPO/editor
Number of trials: 1/1 (1 RUNNING)
+-----------------------+----------+----------------+
| Trial name            | status   | loc            |
|-----------------------+----------+----------------|
| PPO_godot_60f66_00000 | RUNNING  | 127.0.0.1:3084 |
+-----------------------+----------+----------------+

 pid=3084) 2023-02-06 15:32:19,531      INFO trainable.py:127 -- Trainable.setup took 48.063 seconds. If your trainable is slow to initialize, consider setting reuse_actors=True to reduce actor creation overheads.
 pid=3084) 2023-02-06 15:32:19,532      WARNING util.py:55 -- Install gputil for GPU system monitoring.
 pid=13392) connection established
 pid=13392) action space {'move': {'action_type': 'continuous', 'size': 2}}
 pid=13392) observation space {'obs': {'size': [37], 'space': 'box'}}
== Status ==
Current time: 2023-02-06 15:32:20 (running for 00:00:52.39)
Memory usage on this node: 13.9/32.0 GiB
Using FIFO scheduling algorithm.
Resources requested: 2.0/12 CPUs, 0/0 GPUs, 0.0/11.75 GiB heap, 0.0/5.87 GiB objects
Result logdir: C:\Users\Jetpackjules\ray_results\PPO/editor
Number of trials: 1/1 (1 RUNNING)
+-----------------------+----------+----------------+
| Trial name            | status   | loc            |
|-----------------------+----------+----------------|
| PPO_godot_60f66_00000 | RUNNING  | 127.0.0.1:3084 |
+-----------------------+----------+----------------+

== Status ==
Current time: 2023-02-06 15:32:25 (running for 00:00:57.44)
Memory usage on this node: 13.9/32.0 GiB
Using FIFO scheduling algorithm.
Resources requested: 2.0/12 CPUs, 0/0 GPUs, 0.0/11.75 GiB heap, 0.0/5.87 GiB objects
Result logdir: C:\Users\Jetpackjules\ray_results\PPO/editor
Number of trials: 1/1 (1 RUNNING)
+-----------------------+----------+----------------+
| Trial name            | status   | loc            |
|-----------------------+----------+----------------|
| PPO_godot_60f66_00000 | RUNNING  | 127.0.0.1:3084 |
+-----------------------+----------+----------------+

== Status ==
Current time: 2023-02-06 15:32:30 (running for 00:01:02.50)
Memory usage on this node: 13.9/32.0 GiB
Using FIFO scheduling algorithm.
Resources requested: 2.0/12 CPUs, 0/0 GPUs, 0.0/11.75 GiB heap, 0.0/5.87 GiB objects
Result logdir: C:\Users\Jetpackjules\ray_results\PPO/editor
Number of trials: 1/1 (1 RUNNING)
+-----------------------+----------+----------------+
| Trial name            | status   | loc            |
|-----------------------+----------+----------------|
| PPO_godot_60f66_00000 | RUNNING  | 127.0.0.1:3084 |
+-----------------------+----------+----------------+

2023-02-06 15:32:31,191 ERROR trial_runner.py:920 -- Trial PPO_godot_60f66_00000: Error processing event.
Traceback (most recent call last):
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\tune\trial_runner.py", line 886, in _process_trial
    results = self.trial_executor.fetch_result(trial)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\tune\ray_trial_executor.py", line 675, in fetch_result
    result = ray.get(trial_future[0], timeout=DEFAULT_GET_TIMEOUT)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\client_mode_hook.py", line 105, in wrapper
    return func(*args, **kwargs)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\worker.py", line 1763, in get
    raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(RuntimeError): ray::PPOTrainer.train() (pid=3084, ip=127.0.0.1, repr=PPOTrainer)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
    return method(self, *_args, **_kwargs)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 1044, in step_attempt
    step_results = self._exec_plan_or_training_iteration_fn()
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
    return method(self, *_args, **_kwargs)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 2032, in _exec_plan_or_training_iteration_fn
    results = next(self.train_exec_impl)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 756, in __next__
    return next(self.built_iterator)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
    for item in it:
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
    for item in it:
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 843, in apply_filter
    for item in it:
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 843, in apply_filter
    for item in it:
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
    for item in it:
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
    for item in it:
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
    for item in it:
  [Previous line repeated 1 more time]
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 876, in apply_flatten
    for item in it:
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
    for item in it:
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
    for item in it:
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
    for item in it:
  [Previous line repeated 1 more time]
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 471, in base_iterator
    yield ray.get(futures, timeout=timeout)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\client_mode_hook.py", line 105, in wrapper
    return func(*args, **kwargs)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\worker.py", line 1763, in get
    raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(ValueError): ray::RolloutWorker.par_iter_next() (pid=13392, ip=127.0.0.1, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x000001968F6EF6D0>)
ValueError: The two structures don't have the same nested structure.

First structure: type=dict str={'obs': [0.088083224826389, 1.24646044921875, 0.101170279085636, -0.99486911296844, 0.151397776285807, 0.134054991576049, 0.102607933250633, 0.0464364129143792, 0.124826517191019, 0.161533527546101, 0.16446912610853, 0.134228749318166, 0, 0, 0.485937032613668, 0.551738292247325, 0.588589161365956, 0.405113014014992, 0, 0, 0, 0.161773131774352, 0.760299149934236, 0.856947383365116, 0.895181690250431, 0.914702149124833, 0.925685616226884, 0.931842894167513, 0.934741930918651, 0.935008766414883, 0.932697364875862, 0.927319990622031, 0.917565268439216, 0.900363346477887, 0.867662962492522, 0.791059459651913, 0.454066336691917]}

Second structure: type=OrderedDict str=OrderedDict([('obs', array([ 0.83141434,  0.95693845, -0.6448113 , -0.9968011 ,  0.70489717,
        0.48645383,  0.81601405,  0.68891144,  0.8622593 , -0.4577487 ,
        0.16517176,  0.5627296 , -0.1547006 , -0.7543828 , -0.21297579,
        0.2706621 , -0.50107604, -0.80227774,  0.58026075, -0.35255522,
        0.76724803,  0.5039584 , -0.66347104, -0.4566935 , -0.82037747,
       -0.05977842,  0.529509  , -0.28611887,  0.62858206,  0.73975134,
        0.1269484 ,  0.52392966, -0.88686603, -0.0761585 ,  0.3493506 ,
        0.47360763,  0.63514084], dtype=float32))])

More specifically: Substructure "type=list str=[0.088083224826389, 1.24646044921875, 0.101170279085636, -0.99486911296844, 0.151397776285807, 0.134054991576049, 0.102607933250633, 0.0464364129143792, 0.124826517191019, 0.161533527546101, 0.16446912610853, 0.134228749318166, 0, 0, 0.485937032613668, 0.551738292247325, 0.588589161365956, 0.405113014014992, 0, 0, 0, 0.161773131774352, 0.760299149934236, 0.856947383365116, 0.895181690250431, 0.914702149124833, 0.925685616226884, 0.931842894167513, 0.934741930918651, 0.935008766414883, 0.932697364875862, 0.927319990622031, 0.917565268439216, 0.900363346477887, 0.867662962492522, 0.791059459651913, 0.454066336691917]" is a sequence, while substructure "type=ndarray str=[ 0.83141434  0.95693845 -0.6448113  -0.9968011   0.70489717  0.48645383
  0.81601405  0.68891144  0.8622593  -0.4577487   0.16517176  0.5627296
 -0.1547006  -0.7543828  -0.21297579  0.2706621  -0.50107604 -0.80227774
  0.58026075 -0.35255522  0.76724803  0.5039584  -0.66347104 -0.4566935
 -0.82037747 -0.05977842  0.529509   -0.28611887  0.62858206  0.73975134
  0.1269484   0.52392966 -0.88686603 -0.0761585   0.3493506   0.47360763
  0.63514084]" is not

During handling of the above exception, another exception occurred:

ray::RolloutWorker.par_iter_next() (pid=13392, ip=127.0.0.1, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x000001968F6EF6D0>)
  File "python\ray\_raylet.pyx", line 633, in ray._raylet.execute_task
  File "python\ray\_raylet.pyx", line 674, in ray._raylet.execute_task
  File "python\ray\_raylet.pyx", line 640, in ray._raylet.execute_task
  File "python\ray\_raylet.pyx", line 644, in ray._raylet.execute_task
  File "python\ray\_raylet.pyx", line 593, in ray._raylet.execute_task.function_executor
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\function_manager.py", line 648, in actor_method_executor
    return method(__ray_actor, *args, **kwargs)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
    return method(self, *_args, **_kwargs)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 1151, in par_iter_next
    return next(self.local_it)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 380, in gen_rollouts
    yield self.sample()
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
    return method(self, *_args, **_kwargs)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 759, in sample
    batches = [self.input_reader.next()]
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 104, in next
    batches = [self.get_data()]
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 266, in get_data
    item = next(self._env_runner)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 634, in _env_runner
    _process_observations(
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 852, in _process_observations
    prep_obs = preprocessor.transform(raw_obs)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\models\preprocessors.py", line 275, in transform
    self.check_shape(observation)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\models\preprocessors.py", line 68, in check_shape
    observation = convert_element_to_space_type(
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\utils\spaces\space_utils.py", line 321, in convert_element_to_space_type
    return tree.map_structure(
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\tree\__init__.py", line 507, in map_structure
    assert_same_structure(structures[0], other, check_types=check_types)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\tree\__init__.py", line 363, in assert_same_structure
    raise type(e)("%s\n"
ValueError: The two structures don't have the same nested structure.

First structure: type=dict str={'obs': [0.088083224826389, 1.24646044921875, 0.101170279085636, -0.99486911296844, 0.151397776285807, 0.134054991576049, 0.102607933250633, 0.0464364129143792, 0.124826517191019, 0.161533527546101, 0.16446912610853, 0.134228749318166, 0, 0, 0.485937032613668, 0.551738292247325, 0.588589161365956, 0.405113014014992, 0, 0, 0, 0.161773131774352, 0.760299149934236, 0.856947383365116, 0.895181690250431, 0.914702149124833, 0.925685616226884, 0.931842894167513, 0.934741930918651, 0.935008766414883, 0.932697364875862, 0.927319990622031, 0.917565268439216, 0.900363346477887, 0.867662962492522, 0.791059459651913, 0.454066336691917]}

Second structure: type=OrderedDict str=OrderedDict([('obs', array([ 0.83141434,  0.95693845, -0.6448113 , -0.9968011 ,  0.70489717,
        0.48645383,  0.81601405,  0.68891144,  0.8622593 , -0.4577487 ,
        0.16517176,  0.5627296 , -0.1547006 , -0.7543828 , -0.21297579,
        0.2706621 , -0.50107604, -0.80227774,  0.58026075, -0.35255522,
        0.76724803,  0.5039584 , -0.66347104, -0.4566935 , -0.82037747,
       -0.05977842,  0.529509  , -0.28611887,  0.62858206,  0.73975134,
        0.1269484 ,  0.52392966, -0.88686603, -0.0761585 ,  0.3493506 ,
        0.47360763,  0.63514084], dtype=float32))])

More specifically: Substructure "type=list str=[0.088083224826389, 1.24646044921875, 0.101170279085636, -0.99486911296844, 0.151397776285807, 0.134054991576049, 0.102607933250633, 0.0464364129143792, 0.124826517191019, 0.161533527546101, 0.16446912610853, 0.134228749318166, 0, 0, 0.485937032613668, 0.551738292247325, 0.588589161365956, 0.405113014014992, 0, 0, 0, 0.161773131774352, 0.760299149934236, 0.856947383365116, 0.895181690250431, 0.914702149124833, 0.925685616226884, 0.931842894167513, 0.934741930918651, 0.935008766414883, 0.932697364875862, 0.927319990622031, 0.917565268439216, 0.900363346477887, 0.867662962492522, 0.791059459651913, 0.454066336691917]" is a sequence, while substructure "type=ndarray str=[ 0.83141434  0.95693845 -0.6448113  -0.9968011   0.70489717  0.48645383
  0.81601405  0.68891144  0.8622593  -0.4577487   0.16517176  0.5627296
 -0.1547006  -0.7543828  -0.21297579  0.2706621  -0.50107604 -0.80227774
  0.58026075 -0.35255522  0.76724803  0.5039584  -0.66347104 -0.4566935
 -0.82037747 -0.05977842  0.529509   -0.28611887  0.62858206  0.73975134
  0.1269484   0.52392966 -0.88686603 -0.0761585   0.3493506   0.47360763
  0.63514084]" is not
Entire first structure:
{'obs': [., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., .]}
Entire second structure:
OrderedDict([('obs', .)])

During handling of the above exception, another exception occurred:

ray::PPOTrainer.train() (pid=3084, ip=127.0.0.1, repr=PPOTrainer)
  File "python\ray\_raylet.pyx", line 633, in ray._raylet.execute_task
  File "python\ray\_raylet.pyx", line 674, in ray._raylet.execute_task
  File "python\ray\_raylet.pyx", line 640, in ray._raylet.execute_task
  File "python\ray\_raylet.pyx", line 644, in ray._raylet.execute_task
  File "python\ray\_raylet.pyx", line 593, in ray._raylet.execute_task.function_executor
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\function_manager.py", line 648, in actor_method_executor
    return method(__ray_actor, *args, **kwargs)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
    return method(self, *_args, **_kwargs)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\tune\trainable.py", line 319, in train
    result = self.step()
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
    return method(self, *_args, **_kwargs)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 972, in step
    self.try_recover_from_step_attempt()
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
    return method(self, *_args, **_kwargs)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 2434, in try_recover_from_step_attempt
    raise RuntimeError(
RuntimeError: Not enough healthy workers remain to continue.
Result for PPO_godot_60f66_00000:
 pid=3084)  date: 2023-02-06_15-32-19
  experiment_id: b63d176c62e9481ca65e160a090f475d
  hostname: DESKTOP-J8BR5AT
  node_ip: 127.0.0.1
  pid: 3084
  timestamp: 1675726339
  trial_id: 60f66_00000

 2023-02-06 15:32:31,183        ERROR trainer.py:970 -- Error in train call, attempting to recover
 pid=3084) Traceback (most recent call last):
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 965, in step
 pid=3084)     step_attempt_results = self.step_attempt()
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
 pid=3084)     return method(self, *_args, **_kwargs)
== Status ==
Current time: 2023-02-06 15:32:31 (running for 00:01:03.04)
Memory usage on this node: 13.9/32.0 GiB
Using FIFO scheduling algorithm.
Resources requested: 0/12 CPUs, 0/0 GPUs, 0.0/11.75 GiB heap, 0.0/5.87 GiB objects
Result logdir: C:\Users\Jetpackjules\ray_results\PPO/editor
Number of trials: 1/1 (1 ERROR)
+-----------------------+----------+----------------+
| Trial name            | status   | loc            |
|-----------------------+----------+----------------|
| PPO_godot_60f66_00000 | ERROR    | 127.0.0.1:3084 |
+-----------------------+----------+----------------+
Number of errored trials: 1
+-----------------------+--------------+----------------------------------------------------------------------------------------------------+
| Trial name            |   # failures | error file                                                                                         |
|-----------------------+--------------+----------------------------------------------------------------------------------------------------|
| PPO_godot_60f66_00000 |            1 | C:\Users\Jetpackjules\ray_results\PPO/editor\PPO_godot_60f66_00000_0_2023-02-06_15-31-28\error.txt |
+-----------------------+--------------+----------------------------------------------------------------------------------------------------+

 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 1044, in step_attempt
 pid=3084)     step_results = self._exec_plan_or_training_iteration_fn()
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
 pid=3084)     return method(self, *_args, **_kwargs)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 2032, in _exec_plan_or_training_iteration_fn
 pid=3084)     results = next(self.train_exec_impl)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 756, in __next__
 pid=3084)     return next(self.built_iterator)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=3084)     for item in it:
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=3084)     for item in it:
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 843, in apply_filter
 pid=3084)     for item in it:
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 843, in apply_filter
 pid=3084)     for item in it:
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=3084)     for item in it:
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=3084)     for item in it:
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=3084)     for item in it:
 pid=3084)   [Previous line repeated 1 more time]
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 876, in apply_flatten
 pid=3084)     for item in it:
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=3084)     for item in it:
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=3084)     for item in it:
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=3084)     for item in it:
 pid=3084)   [Previous line repeated 1 more time]
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 471, in base_iterator
 pid=3084)     yield ray.get(futures, timeout=timeout)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\client_mode_hook.py", line 105, in wrapper
 pid=3084)     return func(*args, **kwargs)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\worker.py", line 1763, in get
 pid=3084)     raise value.as_instanceof_cause()
 pid=3084) ray.exceptions.RayTaskError(ValueError): ray::RolloutWorker.par_iter_next() (pid=13392, ip=127.0.0.1, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x000001968F6EF6D0>)
 pid=3084) ValueError: The two structures don't have the same nested structure.
 pid=3084)
 pid=3084) First structure: type=dict str={'obs': [0.088083224826389, 1.24646044921875, 0.101170279085636, -0.99486911296844, 0.151397776285807, 0.134054991576049, 0.102607933250633, 0.0464364129143792, 0.124826517191019, 0.161533527546101, 0.16446912610853, 0.134228749318166, 0, 0, 0.485937032613668, 0.551738292247325, 0.588589161365956, 0.405113014014992, 0, 0, 0, 0.161773131774352, 0.760299149934236, 0.856947383365116, 0.895181690250431, 0.914702149124833, 0.925685616226884, 0.931842894167513, 0.934741930918651, 0.935008766414883, 0.932697364875862, 0.927319990622031, 0.917565268439216, 0.900363346477887, 0.867662962492522, 0.791059459651913, 0.454066336691917]}
 pid=3084)
 pid=3084) Second structure: type=OrderedDict str=OrderedDict([('obs', array([ 0.83141434,  0.95693845, -0.6448113 , -0.9968011 ,  0.70489717,
 pid=3084)         0.48645383,  0.81601405,  0.68891144,  0.8622593 , -0.4577487 ,
 pid=3084)         0.16517176,  0.5627296 , -0.1547006 , -0.7543828 , -0.21297579,
 pid=3084)         0.2706621 , -0.50107604, -0.80227774,  0.58026075, -0.35255522,
 pid=3084)         0.76724803,  0.5039584 , -0.66347104, -0.4566935 , -0.82037747,
 pid=3084)        -0.05977842,  0.529509  , -0.28611887,  0.62858206,  0.73975134,
 pid=3084)         0.1269484 ,  0.52392966, -0.88686603, -0.0761585 ,  0.3493506 ,
 pid=3084)         0.47360763,  0.63514084], dtype=float32))])
 pid=3084)
 pid=3084) More specifically: Substructure "type=list str=[0.088083224826389, 1.24646044921875, 0.101170279085636, -0.99486911296844, 0.151397776285807, 0.134054991576049, 0.102607933250633, 0.0464364129143792, 0.124826517191019, 0.161533527546101, 0.16446912610853, 0.134228749318166, 0, 0, 0.485937032613668, 0.551738292247325, 0.588589161365956, 0.405113014014992, 0, 0, 0, 0.161773131774352, 0.760299149934236, 0.856947383365116, 0.895181690250431, 0.914702149124833, 0.925685616226884, 0.931842894167513, 0.934741930918651, 0.935008766414883, 0.932697364875862, 0.927319990622031, 0.917565268439216, 0.900363346477887, 0.867662962492522, 0.791059459651913, 0.454066336691917]" is a sequence, while substructure "type=ndarray str=[ 0.83141434  0.95693845 -0.6448113  -0.9968011   0.70489717  0.48645383
 pid=3084)   0.81601405  0.68891144  0.8622593  -0.4577487   0.16517176  0.5627296
 pid=3084)  -0.1547006  -0.7543828  -0.21297579  0.2706621  -0.50107604 -0.80227774
 pid=3084)   0.58026075 -0.35255522  0.76724803  0.5039584  -0.66347104 -0.4566935
 pid=3084)  -0.82037747 -0.05977842  0.529509   -0.28611887  0.62858206  0.73975134
 pid=3084)   0.1269484   0.52392966 -0.88686603 -0.0761585   0.3493506   0.47360763
 pid=3084)   0.63514084]" is not
 pid=3084)
 pid=3084) During handling of the above exception, another exception occurred:
 pid=3084)
 pid=3084) ray::RolloutWorker.par_iter_next() (pid=13392, ip=127.0.0.1, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x000001968F6EF6D0>)
 pid=3084)   File "python\ray\_raylet.pyx", line 633, in ray._raylet.execute_task
 pid=3084)   File "python\ray\_raylet.pyx", line 674, in ray._raylet.execute_task
 pid=3084)   File "python\ray\_raylet.pyx", line 640, in ray._raylet.execute_task
 pid=3084)   File "python\ray\_raylet.pyx", line 644, in ray._raylet.execute_task
 pid=3084)   File "python\ray\_raylet.pyx", line 593, in ray._raylet.execute_task.function_executor
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\function_manager.py", line 648, in actor_method_executor
 pid=3084)     return method(__ray_actor, *args, **kwargs)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
 pid=3084)     return method(self, *_args, **_kwargs)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 1151, in par_iter_next
 pid=3084)     return next(self.local_it)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 380, in gen_rollouts
 pid=3084)     yield self.sample()
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
 pid=3084)     return method(self, *_args, **_kwargs)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 759, in sample
 pid=3084)     batches = [self.input_reader.next()]
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 104, in next
 pid=3084)     batches = [self.get_data()]
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 266, in get_data
 pid=3084)     item = next(self._env_runner)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 634, in _env_runner
 pid=3084)     _process_observations(
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 852, in _process_observations
 pid=3084)     prep_obs = preprocessor.transform(raw_obs)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\models\preprocessors.py", line 275, in transform
 pid=3084)     self.check_shape(observation)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\models\preprocessors.py", line 68, in check_shape
 pid=3084)     observation = convert_element_to_space_type(
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\utils\spaces\space_utils.py", line 321, in convert_element_to_space_type
 pid=3084)     return tree.map_structure(
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\tree\__init__.py", line 507, in map_structure
 pid=3084)     assert_same_structure(structures[0], other, check_types=check_types)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\tree\__init__.py", line 363, in assert_same_structure
 pid=3084)     raise type(e)("%s\n"
 pid=3084) ValueError: The two structures don't have the same nested structure.
 pid=3084)
 pid=3084) First structure: type=dict str={'obs': [0.088083224826389, 1.24646044921875, 0.101170279085636, -0.99486911296844, 0.151397776285807, 0.134054991576049, 0.102607933250633, 0.0464364129143792, 0.124826517191019, 0.161533527546101, 0.16446912610853, 0.134228749318166, 0, 0, 0.485937032613668, 0.551738292247325, 0.588589161365956, 0.405113014014992, 0, 0, 0, 0.161773131774352, 0.760299149934236, 0.856947383365116, 0.895181690250431, 0.914702149124833, 0.925685616226884, 0.931842894167513, 0.934741930918651, 0.935008766414883, 0.932697364875862, 0.927319990622031, 0.917565268439216, 0.900363346477887, 0.867662962492522, 0.791059459651913, 0.454066336691917]}
 pid=3084)
 pid=3084) Second structure: type=OrderedDict str=OrderedDict([('obs', array([ 0.83141434,  0.95693845, -0.6448113 , -0.9968011 ,  0.70489717,
 pid=3084)         0.48645383,  0.81601405,  0.68891144,  0.8622593 , -0.4577487 ,
 pid=3084)         0.16517176,  0.5627296 , -0.1547006 , -0.7543828 , -0.21297579,
 pid=3084)         0.2706621 , -0.50107604, -0.80227774,  0.58026075, -0.35255522,
 pid=3084)         0.76724803,  0.5039584 , -0.66347104, -0.4566935 , -0.82037747,
 pid=3084)        -0.05977842,  0.529509  , -0.28611887,  0.62858206,  0.73975134,
 pid=3084)         0.1269484 ,  0.52392966, -0.88686603, -0.0761585 ,  0.3493506 ,
 pid=3084)         0.47360763,  0.63514084], dtype=float32))])
 pid=3084)
 pid=3084) More specifically: Substructure "type=list str=[0.088083224826389, 1.24646044921875, 0.101170279085636, -0.99486911296844, 0.151397776285807, 0.134054991576049, 0.102607933250633, 0.0464364129143792, 0.124826517191019, 0.161533527546101, 0.16446912610853, 0.134228749318166, 0, 0, 0.485937032613668, 0.551738292247325, 0.588589161365956, 0.405113014014992, 0, 0, 0, 0.161773131774352, 0.760299149934236, 0.856947383365116, 0.895181690250431, 0.914702149124833, 0.925685616226884, 0.931842894167513, 0.934741930918651, 0.935008766414883, 0.932697364875862, 0.927319990622031, 0.917565268439216, 0.900363346477887, 0.867662962492522, 0.791059459651913, 0.454066336691917]" is a sequence, while substructure "type=ndarray str=[ 0.83141434  0.95693845 -0.6448113  -0.9968011   0.70489717  0.48645383
 pid=3084)   0.81601405  0.68891144  0.8622593  -0.4577487   0.16517176  0.5627296
 pid=3084)  -0.1547006  -0.7543828  -0.21297579  0.2706621  -0.50107604 -0.80227774
 pid=3084)   0.58026075 -0.35255522  0.76724803  0.5039584  -0.66347104 -0.4566935
 pid=3084)  -0.82037747 -0.05977842  0.529509   -0.28611887  0.62858206  0.73975134
 pid=3084)   0.1269484   0.52392966 -0.88686603 -0.0761585   0.3493506   0.47360763
 pid=3084)   0.63514084]" is not
 pid=3084) Entire first structure:
 pid=3084) {'obs': [., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., .]}
 pid=3084) Entire second structure:
 pid=3084) OrderedDict([('obs', .)])
 pid=3084) 2023-02-06 15:32:31,187      ERROR trainer.py:2427 -- Removing unhealthy worker 1
 pid=3084) Traceback (most recent call last):
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 965, in step
 pid=3084)     step_attempt_results = self.step_attempt()
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
 pid=3084)     return method(self, *_args, **_kwargs)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 1044, in step_attempt
 pid=3084)     step_results = self._exec_plan_or_training_iteration_fn()
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
 pid=3084)     return method(self, *_args, **_kwargs)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 2032, in _exec_plan_or_training_iteration_fn
 pid=3084)     results = next(self.train_exec_impl)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 756, in __next__
 pid=3084)     return next(self.built_iterator)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=3084)     for item in it:
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=3084)     for item in it:
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 843, in apply_filter
 pid=3084)     for item in it:
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 843, in apply_filter
 pid=3084)     for item in it:
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=3084)     for item in it:
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=3084)     for item in it:
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=3084)     for item in it:
 pid=3084)   [Previous line repeated 1 more time]
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 876, in apply_flatten
 pid=3084)     for item in it:
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=3084)     for item in it:
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=3084)     for item in it:
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=3084)     for item in it:
 pid=3084)   [Previous line repeated 1 more time]
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 471, in base_iterator
 pid=3084)     yield ray.get(futures, timeout=timeout)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\client_mode_hook.py", line 105, in wrapper
 pid=3084)     return func(*args, **kwargs)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\worker.py", line 1763, in get
 pid=3084)     raise value.as_instanceof_cause()
 pid=3084) ray.exceptions.RayTaskError(ValueError): ray::RolloutWorker.par_iter_next() (pid=13392, ip=127.0.0.1, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x000001968F6EF6D0>)
 pid=3084) ValueError: The two structures don't have the same nested structure.
 pid=3084)
 pid=3084) First structure: type=dict str={'obs': [0.088083224826389, 1.24646044921875, 0.101170279085636, -0.99486911296844, 0.151397776285807, 0.134054991576049, 0.102607933250633, 0.0464364129143792, 0.124826517191019, 0.161533527546101, 0.16446912610853, 0.134228749318166, 0, 0, 0.485937032613668, 0.551738292247325, 0.588589161365956, 0.405113014014992, 0, 0, 0, 0.161773131774352, 0.760299149934236, 0.856947383365116, 0.895181690250431, 0.914702149124833, 0.925685616226884, 0.931842894167513, 0.934741930918651, 0.935008766414883, 0.932697364875862, 0.927319990622031, 0.917565268439216, 0.900363346477887, 0.867662962492522, 0.791059459651913, 0.454066336691917]}
 pid=3084)
 pid=3084) Second structure: type=OrderedDict str=OrderedDict([('obs', array([ 0.83141434,  0.95693845, -0.6448113 , -0.9968011 ,  0.70489717,
 pid=3084)         0.48645383,  0.81601405,  0.68891144,  0.8622593 , -0.4577487 ,
 pid=3084)         0.16517176,  0.5627296 , -0.1547006 , -0.7543828 , -0.21297579,
 pid=3084)         0.2706621 , -0.50107604, -0.80227774,  0.58026075, -0.35255522,
 pid=3084)         0.76724803,  0.5039584 , -0.66347104, -0.4566935 , -0.82037747,
 pid=3084)        -0.05977842,  0.529509  , -0.28611887,  0.62858206,  0.73975134,
 pid=3084)         0.1269484 ,  0.52392966, -0.88686603, -0.0761585 ,  0.3493506 ,
 pid=3084)         0.47360763,  0.63514084], dtype=float32))])
 pid=3084)
 pid=3084) More specifically: Substructure "type=list str=[0.088083224826389, 1.24646044921875, 0.101170279085636, -0.99486911296844, 0.151397776285807, 0.134054991576049, 0.102607933250633, 0.0464364129143792, 0.124826517191019, 0.161533527546101, 0.16446912610853, 0.134228749318166, 0, 0, 0.485937032613668, 0.551738292247325, 0.588589161365956, 0.405113014014992, 0, 0, 0, 0.161773131774352, 0.760299149934236, 0.856947383365116, 0.895181690250431, 0.914702149124833, 0.925685616226884, 0.931842894167513, 0.934741930918651, 0.935008766414883, 0.932697364875862, 0.927319990622031, 0.917565268439216, 0.900363346477887, 0.867662962492522, 0.791059459651913, 0.454066336691917]" is a sequence, while substructure "type=ndarray str=[ 0.83141434  0.95693845 -0.6448113  -0.9968011   0.70489717  0.48645383
 pid=3084)   0.81601405  0.68891144  0.8622593  -0.4577487   0.16517176  0.5627296
 pid=3084)  -0.1547006  -0.7543828  -0.21297579  0.2706621  -0.50107604 -0.80227774
 pid=3084)   0.58026075 -0.35255522  0.76724803  0.5039584  -0.66347104 -0.4566935
 pid=3084)  -0.82037747 -0.05977842  0.529509   -0.28611887  0.62858206  0.73975134
 pid=3084)   0.1269484   0.52392966 -0.88686603 -0.0761585   0.3493506   0.47360763
 pid=3084)   0.63514084]" is not
 pid=3084)
 pid=3084) During handling of the above exception, another exception occurred:
 pid=3084)
 pid=3084) ray::RolloutWorker.par_iter_next() (pid=13392, ip=127.0.0.1, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x000001968F6EF6D0>)
 pid=3084)   File "python\ray\_raylet.pyx", line 633, in ray._raylet.execute_task
 pid=3084)   File "python\ray\_raylet.pyx", line 674, in ray._raylet.execute_task
 pid=3084)   File "python\ray\_raylet.pyx", line 640, in ray._raylet.execute_task
 pid=3084)   File "python\ray\_raylet.pyx", line 644, in ray._raylet.execute_task
 pid=3084)   File "python\ray\_raylet.pyx", line 593, in ray._raylet.execute_task.function_executor
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\function_manager.py", line 648, in actor_method_executor
 pid=3084)     return method(__ray_actor, *args, **kwargs)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
 pid=3084)     return method(self, *_args, **_kwargs)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 1151, in par_iter_next
 pid=3084)     return next(self.local_it)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 380, in gen_rollouts
 pid=3084)     yield self.sample()
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
 pid=3084)     return method(self, *_args, **_kwargs)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 759, in sample
 pid=3084)     batches = [self.input_reader.next()]
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 104, in next
 pid=3084)     batches = [self.get_data()]
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 266, in get_data
 pid=3084)     item = next(self._env_runner)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 634, in _env_runner
 pid=3084)     _process_observations(
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 852, in _process_observations
 pid=3084)     prep_obs = preprocessor.transform(raw_obs)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\models\preprocessors.py", line 275, in transform
 pid=3084)     self.check_shape(observation)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\models\preprocessors.py", line 68, in check_shape
 pid=3084)     observation = convert_element_to_space_type(
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\utils\spaces\space_utils.py", line 321, in convert_element_to_space_type
 pid=3084)     return tree.map_structure(
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\tree\__init__.py", line 507, in map_structure
 pid=3084)     assert_same_structure(structures[0], other, check_types=check_types)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\tree\__init__.py", line 363, in assert_same_structure
 pid=3084)     raise type(e)("%s\n"
 pid=3084) ValueError: The two structures don't have the same nested structure.
 pid=3084)
 pid=3084) First structure: type=dict str={'obs': [0.088083224826389, 1.24646044921875, 0.101170279085636, -0.99486911296844, 0.151397776285807, 0.134054991576049, 0.102607933250633, 0.0464364129143792, 0.124826517191019, 0.161533527546101, 0.16446912610853, 0.134228749318166, 0, 0, 0.485937032613668, 0.551738292247325, 0.588589161365956, 0.405113014014992, 0, 0, 0, 0.161773131774352, 0.760299149934236, 0.856947383365116, 0.895181690250431, 0.914702149124833, 0.925685616226884, 0.931842894167513, 0.934741930918651, 0.935008766414883, 0.932697364875862, 0.927319990622031, 0.917565268439216, 0.900363346477887, 0.867662962492522, 0.791059459651913, 0.454066336691917]}
 pid=3084)
 pid=3084) Second structure: type=OrderedDict str=OrderedDict([('obs', array([ 0.83141434,  0.95693845, -0.6448113 , -0.9968011 ,  0.70489717,
 pid=3084)         0.48645383,  0.81601405,  0.68891144,  0.8622593 , -0.4577487 ,
 pid=3084)         0.16517176,  0.5627296 , -0.1547006 , -0.7543828 , -0.21297579,
 pid=3084)         0.2706621 , -0.50107604, -0.80227774,  0.58026075, -0.35255522,
 pid=3084)         0.76724803,  0.5039584 , -0.66347104, -0.4566935 , -0.82037747,
 pid=3084)        -0.05977842,  0.529509  , -0.28611887,  0.62858206,  0.73975134,
 pid=3084)         0.1269484 ,  0.52392966, -0.88686603, -0.0761585 ,  0.3493506 ,
 pid=3084)         0.47360763,  0.63514084], dtype=float32))])
 pid=3084)
 pid=3084) More specifically: Substructure "type=list str=[0.088083224826389, 1.24646044921875, 0.101170279085636, -0.99486911296844, 0.151397776285807, 0.134054991576049, 0.102607933250633, 0.0464364129143792, 0.124826517191019, 0.161533527546101, 0.16446912610853, 0.134228749318166, 0, 0, 0.485937032613668, 0.551738292247325, 0.588589161365956, 0.405113014014992, 0, 0, 0, 0.161773131774352, 0.760299149934236, 0.856947383365116, 0.895181690250431, 0.914702149124833, 0.925685616226884, 0.931842894167513, 0.934741930918651, 0.935008766414883, 0.932697364875862, 0.927319990622031, 0.917565268439216, 0.900363346477887, 0.867662962492522, 0.791059459651913, 0.454066336691917]" is a sequence, while substructure "type=ndarray str=[ 0.83141434  0.95693845 -0.6448113  -0.9968011   0.70489717  0.48645383
 pid=3084)   0.81601405  0.68891144  0.8622593  -0.4577487   0.16517176  0.5627296
 pid=3084)  -0.1547006  -0.7543828  -0.21297579  0.2706621  -0.50107604 -0.80227774
 pid=3084)   0.58026075 -0.35255522  0.76724803  0.5039584  -0.66347104 -0.4566935
 pid=3084)  -0.82037747 -0.05977842  0.529509   -0.28611887  0.62858206  0.73975134
 pid=3084)   0.1269484   0.52392966 -0.88686603 -0.0761585   0.3493506   0.47360763
 pid=3084)   0.63514084]" is not
 pid=3084) Entire first structure:
 pid=3084) {'obs': [., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., .]}
 pid=3084) Entire second structure:
 pid=3084) OrderedDict([('obs', .)])
 pid=3084)
 pid=3084) During handling of the above exception, another exception occurred:
 pid=3084)
 pid=3084) Traceback (most recent call last):
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 2423, in try_recover_from_step_attempt
 pid=3084)     ray.get(obj_ref)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\client_mode_hook.py", line 105, in wrapper
 pid=3084)     return func(*args, **kwargs)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\worker.py", line 1763, in get
 pid=3084)     raise value.as_instanceof_cause()
 pid=3084) ray.exceptions.RayTaskError(StopIteration): ray::RolloutWorker.sample_with_count() (pid=13392, ip=127.0.0.1, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x000001968F6EF6D0>)
 pid=3084)   File "python\ray\_raylet.pyx", line 640, in ray._raylet.execute_task
 pid=3084)   File "python\ray\_raylet.pyx", line 644, in ray._raylet.execute_task
 pid=3084)   File "python\ray\_raylet.pyx", line 593, in ray._raylet.execute_task.function_executor
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\function_manager.py", line 648, in actor_method_executor
 pid=3084)     return method(__ray_actor, *args, **kwargs)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
 pid=3084)     return method(self, *_args, **_kwargs)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 817, in sample_with_count
 pid=3084)     batch = self.sample()
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
 pid=3084)     return method(self, *_args, **_kwargs)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 759, in sample
 pid=3084)     batches = [self.input_reader.next()]
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 104, in next
 pid=3084)     batches = [self.get_data()]
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 266, in get_data
 pid=3084)     item = next(self._env_runner)
 pid=3084) StopIteration
 pid=3084) 2023-02-06 15:32:31,190      ERROR worker.py:83 -- Unhandled error (suppress with RAY_IGNORE_UNHANDLED_ERRORS=1): ray::RolloutWorker.sample_with_count() (pid=13392, ip=127.0.0.1, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x000001968F6EF6D0>)
 pid=3084)   File "python\ray\_raylet.pyx", line 640, in ray._raylet.execute_task
 pid=3084)   File "python\ray\_raylet.pyx", line 644, in ray._raylet.execute_task
 pid=3084)   File "python\ray\_raylet.pyx", line 593, in ray._raylet.execute_task.function_executor
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\function_manager.py", line 648, in actor_method_executor
 pid=3084)     return method(__ray_actor, *args, **kwargs)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
 pid=3084)     return method(self, *_args, **_kwargs)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 817, in sample_with_count
 pid=3084)     batch = self.sample()
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
 pid=3084)     return method(self, *_args, **_kwargs)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 759, in sample
 pid=3084)     batches = [self.input_reader.next()]
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 104, in next
 pid=3084)     batches = [self.get_data()]
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 266, in get_data
 pid=3084)     item = next(self._env_runner)
 pid=3084) StopIteration
 pid=3084) 2023-02-06 15:32:31,647      ERROR worker_set.py:219 -- Failed to stop workers!
 pid=3084) Traceback (most recent call last):
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\worker_set.py", line 217, in stop
 pid=3084)     ray.get(tids)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\client_mode_hook.py", line 105, in wrapper
 pid=3084)     return func(*args, **kwargs)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\worker.py", line 1765, in get
 pid=3084)     raise value
 pid=3084) ray.exceptions.RayActorError: The actor died unexpectedly before finishing this task.
 pid=3084)      class_name: RolloutWorker
 pid=3084)      actor_id: def0d4155abb49949005ca3e01000000
 pid=3084)      pid: 13392
 pid=3084)      namespace: c2206530-7b97-4281-879d-362dbf58ca34
 pid=3084)      ip: 127.0.0.1
 pid=3084) The actor is dead because its worker process has died. Worker exit type: INTENDED_EXIT
Traceback (most recent call last):
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\Scripts\gdrl.exe\__main__.py", line 7, in <module>
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\godot_rl_agents\core\main.py", line 80, in main
    training_function(args)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\godot_rl_agents\wrappers\ray_wrapper.py", line 103, in rllib_training
    results = tune.run(
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\tune\tune.py", line 633, in run
    raise TuneError("Trials did not complete", incomplete_trials)
ray.tune.error.TuneError: ('Trials did not complete', [PPO_godot_60f66_00000])
 pid=13392) exit was not clean, using atexit to close env
 pid=13392) close message sent
(gdrl_conda) PS C:\Users\Jetpackjules\Documents\Godot Gamez\ANTv0.61>

Any ideas as to what could be causing this array imbalance? My theory is that it has something to do with

func _send_dict_as_json_message(dict):
    stream.put_string(JSON.stringify(dict))

in the sync.gd node, but I don't understand what excatly it would do... (Because the error messages is comparing a list to a dict as incompatible...