search

edbeeching / godot_rl_agents

An Open Source package that allows video game creators, AI researchers and hobbyists the opportunity to learn complex behaviors for their Non Player Characters or agents
MIT License
942 stars 69 forks source link

Code runs for a bit then crashes. (again) #66

Closed Jetpackjules closed 1 year ago

Jetpackjules commented 1 year ago

Unfortunately, It's happening again :(. It sometimes works fine for like an hour, and sometimes I can't for the life of me get it to go more than 5 seconds....

Here is the error I get:

(gdrl_conda) PS C:\Users\Jetpackjules\Documents\Godot Gamez\ANTv0.61> gdrl
run_name PPO/editor
SETTING WORKERS TO 1
{'algorithm': 'PPO', 'RAY_IGNORE_UNHANDLED_ERRORS': 1, 'reuse_actors': True, 'stop': {'episode_reward_mean': 50000, 'training_iteration': 1000000, 'timesteps_total': 2000000000}, 'config': {'env': 'godot', 'env_config': {'framerate': None, 'action_repeat': None, 'show_window': True, 'seed': 0, 'env_path': None}, 'framework': 'torch', 'lambda': 0.95, 'gamma': 0.95, 'ignore_worker_failures': True, 'vf_clip_param': 200.0, 'clip_param': 0.2, 'entropy_coeff': 0.001, 'entropy_coeff_schedule': None, 'train_batch_size': 1024, 'sgd_minibatch_size': 128, 'num_sgd_iter': 16, 'num_workers': 1, 'lr': 0.0003, 'num_envs_per_worker': 1, 'batch_mode': 'truncate_episodes', 'rollout_fragment_length': 16, 'num_gpus': 0, 'model': {'fcnet_hiddens': [256, 256], 'use_lstm': False, 'lstm_cell_size': 32, 'framestack': 4}, 'no_done_at_end': False, 'soft_horizon': False}}
2023-02-06 15:31:28,321 ERROR syncer.py:111 -- Log sync requires rsync to be installed.
 pid=3084) 2023-02-06 15:31:31,471      INFO ppo.py:249 -- In multi-agent mode, policies will be optimized sequentially by the multi-GPU optimizer. Consider setting simple_optimizer=True if this doesn't work for you.
 pid=3084) 2023-02-06 15:31:31,471      INFO trainer.py:779 -- Current log_level is WARN. For more information, set 'log_level': 'INFO' / 'DEBUG' or use the -v and -vv flags.
 pid=13392) No game binary has been provided, please press PLAY in the Godot editor
 pid=13392) waiting for remote GODOT connection on port 11008
== Status ==
Current time: 2023-02-06 15:32:19 (running for 00:00:51.37)
Memory usage on this node: 13.9/32.0 GiB
Using FIFO scheduling algorithm.
Resources requested: 2.0/12 CPUs, 0/0 GPUs, 0.0/11.75 GiB heap, 0.0/5.87 GiB objects
Result logdir: C:\Users\Jetpackjules\ray_results\PPO/editor
Number of trials: 1/1 (1 RUNNING)
+-----------------------+----------+----------------+
| Trial name            | status   | loc            |
|-----------------------+----------+----------------|
| PPO_godot_60f66_00000 | RUNNING  | 127.0.0.1:3084 |
+-----------------------+----------+----------------+

 pid=3084) 2023-02-06 15:32:19,531      INFO trainable.py:127 -- Trainable.setup took 48.063 seconds. If your trainable is slow to initialize, consider setting reuse_actors=True to reduce actor creation overheads.
 pid=3084) 2023-02-06 15:32:19,532      WARNING util.py:55 -- Install gputil for GPU system monitoring.
 pid=13392) connection established
 pid=13392) action space {'move': {'action_type': 'continuous', 'size': 2}}
 pid=13392) observation space {'obs': {'size': [37], 'space': 'box'}}
== Status ==
Current time: 2023-02-06 15:32:20 (running for 00:00:52.39)
Memory usage on this node: 13.9/32.0 GiB
Using FIFO scheduling algorithm.
Resources requested: 2.0/12 CPUs, 0/0 GPUs, 0.0/11.75 GiB heap, 0.0/5.87 GiB objects
Result logdir: C:\Users\Jetpackjules\ray_results\PPO/editor
Number of trials: 1/1 (1 RUNNING)
+-----------------------+----------+----------------+
| Trial name            | status   | loc            |
|-----------------------+----------+----------------|
| PPO_godot_60f66_00000 | RUNNING  | 127.0.0.1:3084 |
+-----------------------+----------+----------------+

== Status ==
Current time: 2023-02-06 15:32:25 (running for 00:00:57.44)
Memory usage on this node: 13.9/32.0 GiB
Using FIFO scheduling algorithm.
Resources requested: 2.0/12 CPUs, 0/0 GPUs, 0.0/11.75 GiB heap, 0.0/5.87 GiB objects
Result logdir: C:\Users\Jetpackjules\ray_results\PPO/editor
Number of trials: 1/1 (1 RUNNING)
+-----------------------+----------+----------------+
| Trial name            | status   | loc            |
|-----------------------+----------+----------------|
| PPO_godot_60f66_00000 | RUNNING  | 127.0.0.1:3084 |
+-----------------------+----------+----------------+

== Status ==
Current time: 2023-02-06 15:32:30 (running for 00:01:02.50)
Memory usage on this node: 13.9/32.0 GiB
Using FIFO scheduling algorithm.
Resources requested: 2.0/12 CPUs, 0/0 GPUs, 0.0/11.75 GiB heap, 0.0/5.87 GiB objects
Result logdir: C:\Users\Jetpackjules\ray_results\PPO/editor
Number of trials: 1/1 (1 RUNNING)
+-----------------------+----------+----------------+
| Trial name            | status   | loc            |
|-----------------------+----------+----------------|
| PPO_godot_60f66_00000 | RUNNING  | 127.0.0.1:3084 |
+-----------------------+----------+----------------+

2023-02-06 15:32:31,191 ERROR trial_runner.py:920 -- Trial PPO_godot_60f66_00000: Error processing event.
Traceback (most recent call last):
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\tune\trial_runner.py", line 886, in _process_trial
    results = self.trial_executor.fetch_result(trial)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\tune\ray_trial_executor.py", line 675, in fetch_result
    result = ray.get(trial_future[0], timeout=DEFAULT_GET_TIMEOUT)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\client_mode_hook.py", line 105, in wrapper
    return func(*args, **kwargs)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\worker.py", line 1763, in get
    raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(RuntimeError): ray::PPOTrainer.train() (pid=3084, ip=127.0.0.1, repr=PPOTrainer)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
    return method(self, *_args, **_kwargs)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 1044, in step_attempt
    step_results = self._exec_plan_or_training_iteration_fn()
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
    return method(self, *_args, **_kwargs)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 2032, in _exec_plan_or_training_iteration_fn
    results = next(self.train_exec_impl)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 756, in __next__
    return next(self.built_iterator)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
    for item in it:
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
    for item in it:
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 843, in apply_filter
    for item in it:
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 843, in apply_filter
    for item in it:
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
    for item in it:
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
    for item in it:
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
    for item in it:
  [Previous line repeated 1 more time]
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 876, in apply_flatten
    for item in it:
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
    for item in it:
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
    for item in it:
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
    for item in it:
  [Previous line repeated 1 more time]
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 471, in base_iterator
    yield ray.get(futures, timeout=timeout)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\client_mode_hook.py", line 105, in wrapper
    return func(*args, **kwargs)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\worker.py", line 1763, in get
    raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(ValueError): ray::RolloutWorker.par_iter_next() (pid=13392, ip=127.0.0.1, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x000001968F6EF6D0>)
ValueError: The two structures don't have the same nested structure.

First structure: type=dict str={'obs': [0.088083224826389, 1.24646044921875, 0.101170279085636, -0.99486911296844, 0.151397776285807, 0.134054991576049, 0.102607933250633, 0.0464364129143792, 0.124826517191019, 0.161533527546101, 0.16446912610853, 0.134228749318166, 0, 0, 0.485937032613668, 0.551738292247325, 0.588589161365956, 0.405113014014992, 0, 0, 0, 0.161773131774352, 0.760299149934236, 0.856947383365116, 0.895181690250431, 0.914702149124833, 0.925685616226884, 0.931842894167513, 0.934741930918651, 0.935008766414883, 0.932697364875862, 0.927319990622031, 0.917565268439216, 0.900363346477887, 0.867662962492522, 0.791059459651913, 0.454066336691917]}

Second structure: type=OrderedDict str=OrderedDict([('obs', array([ 0.83141434,  0.95693845, -0.6448113 , -0.9968011 ,  0.70489717,
        0.48645383,  0.81601405,  0.68891144,  0.8622593 , -0.4577487 ,
        0.16517176,  0.5627296 , -0.1547006 , -0.7543828 , -0.21297579,
        0.2706621 , -0.50107604, -0.80227774,  0.58026075, -0.35255522,
        0.76724803,  0.5039584 , -0.66347104, -0.4566935 , -0.82037747,
       -0.05977842,  0.529509  , -0.28611887,  0.62858206,  0.73975134,
        0.1269484 ,  0.52392966, -0.88686603, -0.0761585 ,  0.3493506 ,
        0.47360763,  0.63514084], dtype=float32))])

More specifically: Substructure "type=list str=[0.088083224826389, 1.24646044921875, 0.101170279085636, -0.99486911296844, 0.151397776285807, 0.134054991576049, 0.102607933250633, 0.0464364129143792, 0.124826517191019, 0.161533527546101, 0.16446912610853, 0.134228749318166, 0, 0, 0.485937032613668, 0.551738292247325, 0.588589161365956, 0.405113014014992, 0, 0, 0, 0.161773131774352, 0.760299149934236, 0.856947383365116, 0.895181690250431, 0.914702149124833, 0.925685616226884, 0.931842894167513, 0.934741930918651, 0.935008766414883, 0.932697364875862, 0.927319990622031, 0.917565268439216, 0.900363346477887, 0.867662962492522, 0.791059459651913, 0.454066336691917]" is a sequence, while substructure "type=ndarray str=[ 0.83141434  0.95693845 -0.6448113  -0.9968011   0.70489717  0.48645383
  0.81601405  0.68891144  0.8622593  -0.4577487   0.16517176  0.5627296
 -0.1547006  -0.7543828  -0.21297579  0.2706621  -0.50107604 -0.80227774
  0.58026075 -0.35255522  0.76724803  0.5039584  -0.66347104 -0.4566935
 -0.82037747 -0.05977842  0.529509   -0.28611887  0.62858206  0.73975134
  0.1269484   0.52392966 -0.88686603 -0.0761585   0.3493506   0.47360763
  0.63514084]" is not

During handling of the above exception, another exception occurred:

ray::RolloutWorker.par_iter_next() (pid=13392, ip=127.0.0.1, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x000001968F6EF6D0>)
  File "python\ray\_raylet.pyx", line 633, in ray._raylet.execute_task
  File "python\ray\_raylet.pyx", line 674, in ray._raylet.execute_task
  File "python\ray\_raylet.pyx", line 640, in ray._raylet.execute_task
  File "python\ray\_raylet.pyx", line 644, in ray._raylet.execute_task
  File "python\ray\_raylet.pyx", line 593, in ray._raylet.execute_task.function_executor
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\function_manager.py", line 648, in actor_method_executor
    return method(__ray_actor, *args, **kwargs)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
    return method(self, *_args, **_kwargs)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 1151, in par_iter_next
    return next(self.local_it)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 380, in gen_rollouts
    yield self.sample()
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
    return method(self, *_args, **_kwargs)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 759, in sample
    batches = [self.input_reader.next()]
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 104, in next
    batches = [self.get_data()]
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 266, in get_data
    item = next(self._env_runner)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 634, in _env_runner
    _process_observations(
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 852, in _process_observations
    prep_obs = preprocessor.transform(raw_obs)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\models\preprocessors.py", line 275, in transform
    self.check_shape(observation)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\models\preprocessors.py", line 68, in check_shape
    observation = convert_element_to_space_type(
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\utils\spaces\space_utils.py", line 321, in convert_element_to_space_type
    return tree.map_structure(
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\tree\__init__.py", line 507, in map_structure
    assert_same_structure(structures[0], other, check_types=check_types)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\tree\__init__.py", line 363, in assert_same_structure
    raise type(e)("%s\n"
ValueError: The two structures don't have the same nested structure.

First structure: type=dict str={'obs': [0.088083224826389, 1.24646044921875, 0.101170279085636, -0.99486911296844, 0.151397776285807, 0.134054991576049, 0.102607933250633, 0.0464364129143792, 0.124826517191019, 0.161533527546101, 0.16446912610853, 0.134228749318166, 0, 0, 0.485937032613668, 0.551738292247325, 0.588589161365956, 0.405113014014992, 0, 0, 0, 0.161773131774352, 0.760299149934236, 0.856947383365116, 0.895181690250431, 0.914702149124833, 0.925685616226884, 0.931842894167513, 0.934741930918651, 0.935008766414883, 0.932697364875862, 0.927319990622031, 0.917565268439216, 0.900363346477887, 0.867662962492522, 0.791059459651913, 0.454066336691917]}

Second structure: type=OrderedDict str=OrderedDict([('obs', array([ 0.83141434,  0.95693845, -0.6448113 , -0.9968011 ,  0.70489717,
        0.48645383,  0.81601405,  0.68891144,  0.8622593 , -0.4577487 ,
        0.16517176,  0.5627296 , -0.1547006 , -0.7543828 , -0.21297579,
        0.2706621 , -0.50107604, -0.80227774,  0.58026075, -0.35255522,
        0.76724803,  0.5039584 , -0.66347104, -0.4566935 , -0.82037747,
       -0.05977842,  0.529509  , -0.28611887,  0.62858206,  0.73975134,
        0.1269484 ,  0.52392966, -0.88686603, -0.0761585 ,  0.3493506 ,
        0.47360763,  0.63514084], dtype=float32))])

More specifically: Substructure "type=list str=[0.088083224826389, 1.24646044921875, 0.101170279085636, -0.99486911296844, 0.151397776285807, 0.134054991576049, 0.102607933250633, 0.0464364129143792, 0.124826517191019, 0.161533527546101, 0.16446912610853, 0.134228749318166, 0, 0, 0.485937032613668, 0.551738292247325, 0.588589161365956, 0.405113014014992, 0, 0, 0, 0.161773131774352, 0.760299149934236, 0.856947383365116, 0.895181690250431, 0.914702149124833, 0.925685616226884, 0.931842894167513, 0.934741930918651, 0.935008766414883, 0.932697364875862, 0.927319990622031, 0.917565268439216, 0.900363346477887, 0.867662962492522, 0.791059459651913, 0.454066336691917]" is a sequence, while substructure "type=ndarray str=[ 0.83141434  0.95693845 -0.6448113  -0.9968011   0.70489717  0.48645383
  0.81601405  0.68891144  0.8622593  -0.4577487   0.16517176  0.5627296
 -0.1547006  -0.7543828  -0.21297579  0.2706621  -0.50107604 -0.80227774
  0.58026075 -0.35255522  0.76724803  0.5039584  -0.66347104 -0.4566935
 -0.82037747 -0.05977842  0.529509   -0.28611887  0.62858206  0.73975134
  0.1269484   0.52392966 -0.88686603 -0.0761585   0.3493506   0.47360763
  0.63514084]" is not
Entire first structure:
{'obs': [., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., .]}
Entire second structure:
OrderedDict([('obs', .)])

During handling of the above exception, another exception occurred:

ray::PPOTrainer.train() (pid=3084, ip=127.0.0.1, repr=PPOTrainer)
  File "python\ray\_raylet.pyx", line 633, in ray._raylet.execute_task
  File "python\ray\_raylet.pyx", line 674, in ray._raylet.execute_task
  File "python\ray\_raylet.pyx", line 640, in ray._raylet.execute_task
  File "python\ray\_raylet.pyx", line 644, in ray._raylet.execute_task
  File "python\ray\_raylet.pyx", line 593, in ray._raylet.execute_task.function_executor
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\function_manager.py", line 648, in actor_method_executor
    return method(__ray_actor, *args, **kwargs)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
    return method(self, *_args, **_kwargs)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\tune\trainable.py", line 319, in train
    result = self.step()
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
    return method(self, *_args, **_kwargs)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 972, in step
    self.try_recover_from_step_attempt()
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
    return method(self, *_args, **_kwargs)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 2434, in try_recover_from_step_attempt
    raise RuntimeError(
RuntimeError: Not enough healthy workers remain to continue.
Result for PPO_godot_60f66_00000:
 pid=3084)  date: 2023-02-06_15-32-19
  experiment_id: b63d176c62e9481ca65e160a090f475d
  hostname: DESKTOP-J8BR5AT
  node_ip: 127.0.0.1
  pid: 3084
  timestamp: 1675726339
  trial_id: 60f66_00000

 2023-02-06 15:32:31,183        ERROR trainer.py:970 -- Error in train call, attempting to recover
 pid=3084) Traceback (most recent call last):
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 965, in step
 pid=3084)     step_attempt_results = self.step_attempt()
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
 pid=3084)     return method(self, *_args, **_kwargs)
== Status ==
Current time: 2023-02-06 15:32:31 (running for 00:01:03.04)
Memory usage on this node: 13.9/32.0 GiB
Using FIFO scheduling algorithm.
Resources requested: 0/12 CPUs, 0/0 GPUs, 0.0/11.75 GiB heap, 0.0/5.87 GiB objects
Result logdir: C:\Users\Jetpackjules\ray_results\PPO/editor
Number of trials: 1/1 (1 ERROR)
+-----------------------+----------+----------------+
| Trial name            | status   | loc            |
|-----------------------+----------+----------------|
| PPO_godot_60f66_00000 | ERROR    | 127.0.0.1:3084 |
+-----------------------+----------+----------------+
Number of errored trials: 1
+-----------------------+--------------+----------------------------------------------------------------------------------------------------+
| Trial name            |   # failures | error file                                                                                         |
|-----------------------+--------------+----------------------------------------------------------------------------------------------------|
| PPO_godot_60f66_00000 |            1 | C:\Users\Jetpackjules\ray_results\PPO/editor\PPO_godot_60f66_00000_0_2023-02-06_15-31-28\error.txt |
+-----------------------+--------------+----------------------------------------------------------------------------------------------------+

 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 1044, in step_attempt
 pid=3084)     step_results = self._exec_plan_or_training_iteration_fn()
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
 pid=3084)     return method(self, *_args, **_kwargs)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 2032, in _exec_plan_or_training_iteration_fn
 pid=3084)     results = next(self.train_exec_impl)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 756, in __next__
 pid=3084)     return next(self.built_iterator)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=3084)     for item in it:
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=3084)     for item in it:
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 843, in apply_filter
 pid=3084)     for item in it:
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 843, in apply_filter
 pid=3084)     for item in it:
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=3084)     for item in it:
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=3084)     for item in it:
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=3084)     for item in it:
 pid=3084)   [Previous line repeated 1 more time]
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 876, in apply_flatten
 pid=3084)     for item in it:
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=3084)     for item in it:
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=3084)     for item in it:
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=3084)     for item in it:
 pid=3084)   [Previous line repeated 1 more time]
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 471, in base_iterator
 pid=3084)     yield ray.get(futures, timeout=timeout)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\client_mode_hook.py", line 105, in wrapper
 pid=3084)     return func(*args, **kwargs)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\worker.py", line 1763, in get
 pid=3084)     raise value.as_instanceof_cause()
 pid=3084) ray.exceptions.RayTaskError(ValueError): ray::RolloutWorker.par_iter_next() (pid=13392, ip=127.0.0.1, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x000001968F6EF6D0>)
 pid=3084) ValueError: The two structures don't have the same nested structure.
 pid=3084)
 pid=3084) First structure: type=dict str={'obs': [0.088083224826389, 1.24646044921875, 0.101170279085636, -0.99486911296844, 0.151397776285807, 0.134054991576049, 0.102607933250633, 0.0464364129143792, 0.124826517191019, 0.161533527546101, 0.16446912610853, 0.134228749318166, 0, 0, 0.485937032613668, 0.551738292247325, 0.588589161365956, 0.405113014014992, 0, 0, 0, 0.161773131774352, 0.760299149934236, 0.856947383365116, 0.895181690250431, 0.914702149124833, 0.925685616226884, 0.931842894167513, 0.934741930918651, 0.935008766414883, 0.932697364875862, 0.927319990622031, 0.917565268439216, 0.900363346477887, 0.867662962492522, 0.791059459651913, 0.454066336691917]}
 pid=3084)
 pid=3084) Second structure: type=OrderedDict str=OrderedDict([('obs', array([ 0.83141434,  0.95693845, -0.6448113 , -0.9968011 ,  0.70489717,
 pid=3084)         0.48645383,  0.81601405,  0.68891144,  0.8622593 , -0.4577487 ,
 pid=3084)         0.16517176,  0.5627296 , -0.1547006 , -0.7543828 , -0.21297579,
 pid=3084)         0.2706621 , -0.50107604, -0.80227774,  0.58026075, -0.35255522,
 pid=3084)         0.76724803,  0.5039584 , -0.66347104, -0.4566935 , -0.82037747,
 pid=3084)        -0.05977842,  0.529509  , -0.28611887,  0.62858206,  0.73975134,
 pid=3084)         0.1269484 ,  0.52392966, -0.88686603, -0.0761585 ,  0.3493506 ,
 pid=3084)         0.47360763,  0.63514084], dtype=float32))])
 pid=3084)
 pid=3084) More specifically: Substructure "type=list str=[0.088083224826389, 1.24646044921875, 0.101170279085636, -0.99486911296844, 0.151397776285807, 0.134054991576049, 0.102607933250633, 0.0464364129143792, 0.124826517191019, 0.161533527546101, 0.16446912610853, 0.134228749318166, 0, 0, 0.485937032613668, 0.551738292247325, 0.588589161365956, 0.405113014014992, 0, 0, 0, 0.161773131774352, 0.760299149934236, 0.856947383365116, 0.895181690250431, 0.914702149124833, 0.925685616226884, 0.931842894167513, 0.934741930918651, 0.935008766414883, 0.932697364875862, 0.927319990622031, 0.917565268439216, 0.900363346477887, 0.867662962492522, 0.791059459651913, 0.454066336691917]" is a sequence, while substructure "type=ndarray str=[ 0.83141434  0.95693845 -0.6448113  -0.9968011   0.70489717  0.48645383
 pid=3084)   0.81601405  0.68891144  0.8622593  -0.4577487   0.16517176  0.5627296
 pid=3084)  -0.1547006  -0.7543828  -0.21297579  0.2706621  -0.50107604 -0.80227774
 pid=3084)   0.58026075 -0.35255522  0.76724803  0.5039584  -0.66347104 -0.4566935
 pid=3084)  -0.82037747 -0.05977842  0.529509   -0.28611887  0.62858206  0.73975134
 pid=3084)   0.1269484   0.52392966 -0.88686603 -0.0761585   0.3493506   0.47360763
 pid=3084)   0.63514084]" is not
 pid=3084)
 pid=3084) During handling of the above exception, another exception occurred:
 pid=3084)
 pid=3084) ray::RolloutWorker.par_iter_next() (pid=13392, ip=127.0.0.1, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x000001968F6EF6D0>)
 pid=3084)   File "python\ray\_raylet.pyx", line 633, in ray._raylet.execute_task
 pid=3084)   File "python\ray\_raylet.pyx", line 674, in ray._raylet.execute_task
 pid=3084)   File "python\ray\_raylet.pyx", line 640, in ray._raylet.execute_task
 pid=3084)   File "python\ray\_raylet.pyx", line 644, in ray._raylet.execute_task
 pid=3084)   File "python\ray\_raylet.pyx", line 593, in ray._raylet.execute_task.function_executor
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\function_manager.py", line 648, in actor_method_executor
 pid=3084)     return method(__ray_actor, *args, **kwargs)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
 pid=3084)     return method(self, *_args, **_kwargs)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 1151, in par_iter_next
 pid=3084)     return next(self.local_it)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 380, in gen_rollouts
 pid=3084)     yield self.sample()
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
 pid=3084)     return method(self, *_args, **_kwargs)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 759, in sample
 pid=3084)     batches = [self.input_reader.next()]
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 104, in next
 pid=3084)     batches = [self.get_data()]
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 266, in get_data
 pid=3084)     item = next(self._env_runner)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 634, in _env_runner
 pid=3084)     _process_observations(
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 852, in _process_observations
 pid=3084)     prep_obs = preprocessor.transform(raw_obs)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\models\preprocessors.py", line 275, in transform
 pid=3084)     self.check_shape(observation)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\models\preprocessors.py", line 68, in check_shape
 pid=3084)     observation = convert_element_to_space_type(
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\utils\spaces\space_utils.py", line 321, in convert_element_to_space_type
 pid=3084)     return tree.map_structure(
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\tree\__init__.py", line 507, in map_structure
 pid=3084)     assert_same_structure(structures[0], other, check_types=check_types)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\tree\__init__.py", line 363, in assert_same_structure
 pid=3084)     raise type(e)("%s\n"
 pid=3084) ValueError: The two structures don't have the same nested structure.
 pid=3084)
 pid=3084) First structure: type=dict str={'obs': [0.088083224826389, 1.24646044921875, 0.101170279085636, -0.99486911296844, 0.151397776285807, 0.134054991576049, 0.102607933250633, 0.0464364129143792, 0.124826517191019, 0.161533527546101, 0.16446912610853, 0.134228749318166, 0, 0, 0.485937032613668, 0.551738292247325, 0.588589161365956, 0.405113014014992, 0, 0, 0, 0.161773131774352, 0.760299149934236, 0.856947383365116, 0.895181690250431, 0.914702149124833, 0.925685616226884, 0.931842894167513, 0.934741930918651, 0.935008766414883, 0.932697364875862, 0.927319990622031, 0.917565268439216, 0.900363346477887, 0.867662962492522, 0.791059459651913, 0.454066336691917]}
 pid=3084)
 pid=3084) Second structure: type=OrderedDict str=OrderedDict([('obs', array([ 0.83141434,  0.95693845, -0.6448113 , -0.9968011 ,  0.70489717,
 pid=3084)         0.48645383,  0.81601405,  0.68891144,  0.8622593 , -0.4577487 ,
 pid=3084)         0.16517176,  0.5627296 , -0.1547006 , -0.7543828 , -0.21297579,
 pid=3084)         0.2706621 , -0.50107604, -0.80227774,  0.58026075, -0.35255522,
 pid=3084)         0.76724803,  0.5039584 , -0.66347104, -0.4566935 , -0.82037747,
 pid=3084)        -0.05977842,  0.529509  , -0.28611887,  0.62858206,  0.73975134,
 pid=3084)         0.1269484 ,  0.52392966, -0.88686603, -0.0761585 ,  0.3493506 ,
 pid=3084)         0.47360763,  0.63514084], dtype=float32))])
 pid=3084)
 pid=3084) More specifically: Substructure "type=list str=[0.088083224826389, 1.24646044921875, 0.101170279085636, -0.99486911296844, 0.151397776285807, 0.134054991576049, 0.102607933250633, 0.0464364129143792, 0.124826517191019, 0.161533527546101, 0.16446912610853, 0.134228749318166, 0, 0, 0.485937032613668, 0.551738292247325, 0.588589161365956, 0.405113014014992, 0, 0, 0, 0.161773131774352, 0.760299149934236, 0.856947383365116, 0.895181690250431, 0.914702149124833, 0.925685616226884, 0.931842894167513, 0.934741930918651, 0.935008766414883, 0.932697364875862, 0.927319990622031, 0.917565268439216, 0.900363346477887, 0.867662962492522, 0.791059459651913, 0.454066336691917]" is a sequence, while substructure "type=ndarray str=[ 0.83141434  0.95693845 -0.6448113  -0.9968011   0.70489717  0.48645383
 pid=3084)   0.81601405  0.68891144  0.8622593  -0.4577487   0.16517176  0.5627296
 pid=3084)  -0.1547006  -0.7543828  -0.21297579  0.2706621  -0.50107604 -0.80227774
 pid=3084)   0.58026075 -0.35255522  0.76724803  0.5039584  -0.66347104 -0.4566935
 pid=3084)  -0.82037747 -0.05977842  0.529509   -0.28611887  0.62858206  0.73975134
 pid=3084)   0.1269484   0.52392966 -0.88686603 -0.0761585   0.3493506   0.47360763
 pid=3084)   0.63514084]" is not
 pid=3084) Entire first structure:
 pid=3084) {'obs': [., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., .]}
 pid=3084) Entire second structure:
 pid=3084) OrderedDict([('obs', .)])
 pid=3084) 2023-02-06 15:32:31,187      ERROR trainer.py:2427 -- Removing unhealthy worker 1
 pid=3084) Traceback (most recent call last):
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 965, in step
 pid=3084)     step_attempt_results = self.step_attempt()
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
 pid=3084)     return method(self, *_args, **_kwargs)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 1044, in step_attempt
 pid=3084)     step_results = self._exec_plan_or_training_iteration_fn()
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
 pid=3084)     return method(self, *_args, **_kwargs)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 2032, in _exec_plan_or_training_iteration_fn
 pid=3084)     results = next(self.train_exec_impl)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 756, in __next__
 pid=3084)     return next(self.built_iterator)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=3084)     for item in it:
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=3084)     for item in it:
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 843, in apply_filter
 pid=3084)     for item in it:
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 843, in apply_filter
 pid=3084)     for item in it:
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=3084)     for item in it:
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=3084)     for item in it:
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=3084)     for item in it:
 pid=3084)   [Previous line repeated 1 more time]
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 876, in apply_flatten
 pid=3084)     for item in it:
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=3084)     for item in it:
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=3084)     for item in it:
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
 pid=3084)     for item in it:
 pid=3084)   [Previous line repeated 1 more time]
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 471, in base_iterator
 pid=3084)     yield ray.get(futures, timeout=timeout)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\client_mode_hook.py", line 105, in wrapper
 pid=3084)     return func(*args, **kwargs)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\worker.py", line 1763, in get
 pid=3084)     raise value.as_instanceof_cause()
 pid=3084) ray.exceptions.RayTaskError(ValueError): ray::RolloutWorker.par_iter_next() (pid=13392, ip=127.0.0.1, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x000001968F6EF6D0>)
 pid=3084) ValueError: The two structures don't have the same nested structure.
 pid=3084)
 pid=3084) First structure: type=dict str={'obs': [0.088083224826389, 1.24646044921875, 0.101170279085636, -0.99486911296844, 0.151397776285807, 0.134054991576049, 0.102607933250633, 0.0464364129143792, 0.124826517191019, 0.161533527546101, 0.16446912610853, 0.134228749318166, 0, 0, 0.485937032613668, 0.551738292247325, 0.588589161365956, 0.405113014014992, 0, 0, 0, 0.161773131774352, 0.760299149934236, 0.856947383365116, 0.895181690250431, 0.914702149124833, 0.925685616226884, 0.931842894167513, 0.934741930918651, 0.935008766414883, 0.932697364875862, 0.927319990622031, 0.917565268439216, 0.900363346477887, 0.867662962492522, 0.791059459651913, 0.454066336691917]}
 pid=3084)
 pid=3084) Second structure: type=OrderedDict str=OrderedDict([('obs', array([ 0.83141434,  0.95693845, -0.6448113 , -0.9968011 ,  0.70489717,
 pid=3084)         0.48645383,  0.81601405,  0.68891144,  0.8622593 , -0.4577487 ,
 pid=3084)         0.16517176,  0.5627296 , -0.1547006 , -0.7543828 , -0.21297579,
 pid=3084)         0.2706621 , -0.50107604, -0.80227774,  0.58026075, -0.35255522,
 pid=3084)         0.76724803,  0.5039584 , -0.66347104, -0.4566935 , -0.82037747,
 pid=3084)        -0.05977842,  0.529509  , -0.28611887,  0.62858206,  0.73975134,
 pid=3084)         0.1269484 ,  0.52392966, -0.88686603, -0.0761585 ,  0.3493506 ,
 pid=3084)         0.47360763,  0.63514084], dtype=float32))])
 pid=3084)
 pid=3084) More specifically: Substructure "type=list str=[0.088083224826389, 1.24646044921875, 0.101170279085636, -0.99486911296844, 0.151397776285807, 0.134054991576049, 0.102607933250633, 0.0464364129143792, 0.124826517191019, 0.161533527546101, 0.16446912610853, 0.134228749318166, 0, 0, 0.485937032613668, 0.551738292247325, 0.588589161365956, 0.405113014014992, 0, 0, 0, 0.161773131774352, 0.760299149934236, 0.856947383365116, 0.895181690250431, 0.914702149124833, 0.925685616226884, 0.931842894167513, 0.934741930918651, 0.935008766414883, 0.932697364875862, 0.927319990622031, 0.917565268439216, 0.900363346477887, 0.867662962492522, 0.791059459651913, 0.454066336691917]" is a sequence, while substructure "type=ndarray str=[ 0.83141434  0.95693845 -0.6448113  -0.9968011   0.70489717  0.48645383
 pid=3084)   0.81601405  0.68891144  0.8622593  -0.4577487   0.16517176  0.5627296
 pid=3084)  -0.1547006  -0.7543828  -0.21297579  0.2706621  -0.50107604 -0.80227774
 pid=3084)   0.58026075 -0.35255522  0.76724803  0.5039584  -0.66347104 -0.4566935
 pid=3084)  -0.82037747 -0.05977842  0.529509   -0.28611887  0.62858206  0.73975134
 pid=3084)   0.1269484   0.52392966 -0.88686603 -0.0761585   0.3493506   0.47360763
 pid=3084)   0.63514084]" is not
 pid=3084)
 pid=3084) During handling of the above exception, another exception occurred:
 pid=3084)
 pid=3084) ray::RolloutWorker.par_iter_next() (pid=13392, ip=127.0.0.1, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x000001968F6EF6D0>)
 pid=3084)   File "python\ray\_raylet.pyx", line 633, in ray._raylet.execute_task
 pid=3084)   File "python\ray\_raylet.pyx", line 674, in ray._raylet.execute_task
 pid=3084)   File "python\ray\_raylet.pyx", line 640, in ray._raylet.execute_task
 pid=3084)   File "python\ray\_raylet.pyx", line 644, in ray._raylet.execute_task
 pid=3084)   File "python\ray\_raylet.pyx", line 593, in ray._raylet.execute_task.function_executor
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\function_manager.py", line 648, in actor_method_executor
 pid=3084)     return method(__ray_actor, *args, **kwargs)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
 pid=3084)     return method(self, *_args, **_kwargs)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 1151, in par_iter_next
 pid=3084)     return next(self.local_it)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 380, in gen_rollouts
 pid=3084)     yield self.sample()
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
 pid=3084)     return method(self, *_args, **_kwargs)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 759, in sample
 pid=3084)     batches = [self.input_reader.next()]
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 104, in next
 pid=3084)     batches = [self.get_data()]
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 266, in get_data
 pid=3084)     item = next(self._env_runner)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 634, in _env_runner
 pid=3084)     _process_observations(
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 852, in _process_observations
 pid=3084)     prep_obs = preprocessor.transform(raw_obs)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\models\preprocessors.py", line 275, in transform
 pid=3084)     self.check_shape(observation)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\models\preprocessors.py", line 68, in check_shape
 pid=3084)     observation = convert_element_to_space_type(
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\utils\spaces\space_utils.py", line 321, in convert_element_to_space_type
 pid=3084)     return tree.map_structure(
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\tree\__init__.py", line 507, in map_structure
 pid=3084)     assert_same_structure(structures[0], other, check_types=check_types)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\tree\__init__.py", line 363, in assert_same_structure
 pid=3084)     raise type(e)("%s\n"
 pid=3084) ValueError: The two structures don't have the same nested structure.
 pid=3084)
 pid=3084) First structure: type=dict str={'obs': [0.088083224826389, 1.24646044921875, 0.101170279085636, -0.99486911296844, 0.151397776285807, 0.134054991576049, 0.102607933250633, 0.0464364129143792, 0.124826517191019, 0.161533527546101, 0.16446912610853, 0.134228749318166, 0, 0, 0.485937032613668, 0.551738292247325, 0.588589161365956, 0.405113014014992, 0, 0, 0, 0.161773131774352, 0.760299149934236, 0.856947383365116, 0.895181690250431, 0.914702149124833, 0.925685616226884, 0.931842894167513, 0.934741930918651, 0.935008766414883, 0.932697364875862, 0.927319990622031, 0.917565268439216, 0.900363346477887, 0.867662962492522, 0.791059459651913, 0.454066336691917]}
 pid=3084)
 pid=3084) Second structure: type=OrderedDict str=OrderedDict([('obs', array([ 0.83141434,  0.95693845, -0.6448113 , -0.9968011 ,  0.70489717,
 pid=3084)         0.48645383,  0.81601405,  0.68891144,  0.8622593 , -0.4577487 ,
 pid=3084)         0.16517176,  0.5627296 , -0.1547006 , -0.7543828 , -0.21297579,
 pid=3084)         0.2706621 , -0.50107604, -0.80227774,  0.58026075, -0.35255522,
 pid=3084)         0.76724803,  0.5039584 , -0.66347104, -0.4566935 , -0.82037747,
 pid=3084)        -0.05977842,  0.529509  , -0.28611887,  0.62858206,  0.73975134,
 pid=3084)         0.1269484 ,  0.52392966, -0.88686603, -0.0761585 ,  0.3493506 ,
 pid=3084)         0.47360763,  0.63514084], dtype=float32))])
 pid=3084)
 pid=3084) More specifically: Substructure "type=list str=[0.088083224826389, 1.24646044921875, 0.101170279085636, -0.99486911296844, 0.151397776285807, 0.134054991576049, 0.102607933250633, 0.0464364129143792, 0.124826517191019, 0.161533527546101, 0.16446912610853, 0.134228749318166, 0, 0, 0.485937032613668, 0.551738292247325, 0.588589161365956, 0.405113014014992, 0, 0, 0, 0.161773131774352, 0.760299149934236, 0.856947383365116, 0.895181690250431, 0.914702149124833, 0.925685616226884, 0.931842894167513, 0.934741930918651, 0.935008766414883, 0.932697364875862, 0.927319990622031, 0.917565268439216, 0.900363346477887, 0.867662962492522, 0.791059459651913, 0.454066336691917]" is a sequence, while substructure "type=ndarray str=[ 0.83141434  0.95693845 -0.6448113  -0.9968011   0.70489717  0.48645383
 pid=3084)   0.81601405  0.68891144  0.8622593  -0.4577487   0.16517176  0.5627296
 pid=3084)  -0.1547006  -0.7543828  -0.21297579  0.2706621  -0.50107604 -0.80227774
 pid=3084)   0.58026075 -0.35255522  0.76724803  0.5039584  -0.66347104 -0.4566935
 pid=3084)  -0.82037747 -0.05977842  0.529509   -0.28611887  0.62858206  0.73975134
 pid=3084)   0.1269484   0.52392966 -0.88686603 -0.0761585   0.3493506   0.47360763
 pid=3084)   0.63514084]" is not
 pid=3084) Entire first structure:
 pid=3084) {'obs': [., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., .]}
 pid=3084) Entire second structure:
 pid=3084) OrderedDict([('obs', .)])
 pid=3084)
 pid=3084) During handling of the above exception, another exception occurred:
 pid=3084)
 pid=3084) Traceback (most recent call last):
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 2423, in try_recover_from_step_attempt
 pid=3084)     ray.get(obj_ref)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\client_mode_hook.py", line 105, in wrapper
 pid=3084)     return func(*args, **kwargs)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\worker.py", line 1763, in get
 pid=3084)     raise value.as_instanceof_cause()
 pid=3084) ray.exceptions.RayTaskError(StopIteration): ray::RolloutWorker.sample_with_count() (pid=13392, ip=127.0.0.1, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x000001968F6EF6D0>)
 pid=3084)   File "python\ray\_raylet.pyx", line 640, in ray._raylet.execute_task
 pid=3084)   File "python\ray\_raylet.pyx", line 644, in ray._raylet.execute_task
 pid=3084)   File "python\ray\_raylet.pyx", line 593, in ray._raylet.execute_task.function_executor
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\function_manager.py", line 648, in actor_method_executor
 pid=3084)     return method(__ray_actor, *args, **kwargs)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
 pid=3084)     return method(self, *_args, **_kwargs)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 817, in sample_with_count
 pid=3084)     batch = self.sample()
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
 pid=3084)     return method(self, *_args, **_kwargs)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 759, in sample
 pid=3084)     batches = [self.input_reader.next()]
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 104, in next
 pid=3084)     batches = [self.get_data()]
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 266, in get_data
 pid=3084)     item = next(self._env_runner)
 pid=3084) StopIteration
 pid=3084) 2023-02-06 15:32:31,190      ERROR worker.py:83 -- Unhandled error (suppress with RAY_IGNORE_UNHANDLED_ERRORS=1): ray::RolloutWorker.sample_with_count() (pid=13392, ip=127.0.0.1, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x000001968F6EF6D0>)
 pid=3084)   File "python\ray\_raylet.pyx", line 640, in ray._raylet.execute_task
 pid=3084)   File "python\ray\_raylet.pyx", line 644, in ray._raylet.execute_task
 pid=3084)   File "python\ray\_raylet.pyx", line 593, in ray._raylet.execute_task.function_executor
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\function_manager.py", line 648, in actor_method_executor
 pid=3084)     return method(__ray_actor, *args, **kwargs)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
 pid=3084)     return method(self, *_args, **_kwargs)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 817, in sample_with_count
 pid=3084)     batch = self.sample()
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
 pid=3084)     return method(self, *_args, **_kwargs)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 759, in sample
 pid=3084)     batches = [self.input_reader.next()]
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 104, in next
 pid=3084)     batches = [self.get_data()]
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 266, in get_data
 pid=3084)     item = next(self._env_runner)
 pid=3084) StopIteration
 pid=3084) 2023-02-06 15:32:31,647      ERROR worker_set.py:219 -- Failed to stop workers!
 pid=3084) Traceback (most recent call last):
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\worker_set.py", line 217, in stop
 pid=3084)     ray.get(tids)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\client_mode_hook.py", line 105, in wrapper
 pid=3084)     return func(*args, **kwargs)
 pid=3084)   File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\worker.py", line 1765, in get
 pid=3084)     raise value
 pid=3084) ray.exceptions.RayActorError: The actor died unexpectedly before finishing this task.
 pid=3084)      class_name: RolloutWorker
 pid=3084)      actor_id: def0d4155abb49949005ca3e01000000
 pid=3084)      pid: 13392
 pid=3084)      namespace: c2206530-7b97-4281-879d-362dbf58ca34
 pid=3084)      ip: 127.0.0.1
 pid=3084) The actor is dead because its worker process has died. Worker exit type: INTENDED_EXIT
Traceback (most recent call last):
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\Scripts\gdrl.exe\__main__.py", line 7, in <module>
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\godot_rl_agents\core\main.py", line 80, in main
    training_function(args)
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\godot_rl_agents\wrappers\ray_wrapper.py", line 103, in rllib_training
    results = tune.run(
  File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\tune\tune.py", line 633, in run
    raise TuneError("Trials did not complete", incomplete_trials)
ray.tune.error.TuneError: ('Trials did not complete', [PPO_godot_60f66_00000])
 pid=13392) exit was not clean, using atexit to close env
 pid=13392) close message sent
(gdrl_conda) PS C:\Users\Jetpackjules\Documents\Godot Gamez\ANTv0.61>

Any ideas as to what could be causing this array imbalance? My theory is that it has something to do with

func _send_dict_as_json_message(dict):
    stream.put_string(JSON.stringify(dict))

in the sync.gd node, but I don't understand what excatly it would do... (Because the error messages is comparing a list to a dict as incompatible...

ryash072007 commented 1 year ago

I am pretty sure that _send_dict_as_json_message is not the problem as I have run sb3 with no problem for two whole days without an error. _send_dict_as_json_message is a part of Godot Env so if it were to cause errors, it would happen to everyone.

Jetpackjules commented 1 year ago

Any other guesses? I can't figure out why this is happening...

ryash072007 commented 1 year ago

Can you explain your environment or make a repo of your godot project, I'll check it out and see for errors.

ryash072007 commented 1 year ago

preferably make repo

Jetpackjules commented 1 year ago

I will try to upload my project tn, thanks for the help.

edbeeching commented 1 year ago

I think the problem lies somewhere in your project rather than the Godot RL interface. How many agents are there in your env and are there always the same number? You do not remove agents when they die?

Jetpackjules commented 1 year ago

Here is the repository: https://github.com/Jetpackjules/GODOT_RL_ANTS

Jetpackjules commented 1 year ago

I think the problem lies somewhere in your project rather than the Godot RL interface. How many agents are there in your env and are there always the same number? You do not remove agents when they die?

Pretty sure I do... but maybe not? not too sure of the specifications of 'die' in this context... do you mean reward reset?

ryash072007 commented 1 year ago

I think the problem lies somewhere in your project rather than the Godot RL interface. How many agents are there in your env and are there always the same number? You do not remove agents when they die?

Pretty sure I do... but maybe not? not too sure of the specifications of 'die' in this context... do you mean reward reset?

the number of agents SHOULD remain a constant at all times and should whenever requested, provide obs and rewards

ryash072007 commented 1 year ago

I tried your project with sb3 and Godot.Beta.16 and it works just fine out of the box. All I had to do was fix one error(Maybe becoz I use Godot 4, not sure tho) which was to move the @tool keyword to the first line of the script in the raycastsensor2d script.

Jetpackjules commented 1 year ago

I tried your project with sb3 and Godot.Beta.16 and it works just fine out of the box. All I had to do was fix one error(Maybe becoz I use Godot 4, not sure tho) which was to move the @tool keyword to the first line of the script in the raycastsensor2d script.

That's good news... were you able to try with rlib using the ppo_test.yaml? That has all my current settings... (I am also using Godot 4)

ryash072007 commented 1 year ago

I tried your project with sb3 and Godot.Beta.16 and it works just fine out of the box. All I had to do was fix one error(Maybe becoz I use Godot 4, not sure tho) which was to move the @tool keyword to the first line of the script in the raycastsensor2d script.

That's good news... were you able to try with rlib using the ppo_test.yaml? That has all my current settings... (I am also using Godot 4)

Unfortunately, RLLIB doesn't work for me, so I can't.

ryash072007 commented 1 year ago

I tried your project with sb3 and Godot.Beta.16 and it works just fine out of the box. All I had to do was fix one error(Maybe becoz I use Godot 4, not sure tho) which was to move the @tool keyword to the first line of the script in the raycastsensor2d script.

That's good news... were you able to try with rlib using the ppo_test.yaml? That has all my current settings... (I am also using Godot 4)

then move the @tool keyword to the top of the script.

Jetpackjules commented 1 year ago

I tried your project with sb3 and Godot.Beta.16 and it works just fine out of the box. All I had to do was fix one error(Maybe becoz I use Godot 4, not sure tho) which was to move the @tool keyword to the first line of the script in the raycastsensor2d script.

That's good news... were you able to try with rlib using the ppo_test.yaml? That has all my current settings... (I am also using Godot 4)

then move the @tool keyword to the top of the script.

Ok, I will try that when I get home.

Jetpackjules commented 1 year ago

I tried your project with sb3 and Godot.Beta.16 and it works just fine out of the box. All I had to do was fix one error(Maybe becoz I use Godot 4, not sure tho) which was to move the @tool keyword to the first line of the script in the raycastsensor2d script.

That's good news... were you able to try with rlib using the ppo_test.yaml? That has all my current settings... (I am also using Godot 4)

then move the @tool keyword to the top of the script.

Unfortunately, that did not fix it, it runs for a bit then crashes with the same error as above :(

Jetpackjules commented 1 year ago

Could I run it on Sb3 on windows? (If so, how can I port my .yaml settings?)

ryash072007 commented 1 year ago

SB3 works good on windows(Even I use windows btw). As for porting your .yaml settings, it will need to be done by code as sb3 doesnt use .yaml settings.

This wrapper for sb3 gives you more options btw: https://github.com/ryash072007/godot_rl_agents_forked/blob/main/godot_rl/backend/sb3.py

Jetpackjules commented 1 year ago

Great, SB3 seems to work! Thanks for the help. Quick question, where are the output logs?

ryash072007 commented 1 year ago

Great, SB3 seems to work! Thanks for the help. Quick question, where are the output logs?

It should be in some "Log" folder, it will be in the directory in which the command prompt was in when gdrl was run or when sb3.py was run.

Jetpackjules commented 1 year ago

Thanks! I will close this for now, as all my problems have been adressed. I appreciate the help.