Ok, so I've gotten the trainer to work (it's currently training with no issues, anyway).
I had to specify the paths at command line with some additional passed arguments. This is what I used (using GeneralExplorationPolicy as base directory):
$ python trainners/runner.py -c trainners/trainnerV2.json -r Level-1
Additionally, in the utils.py file, I had to uncomment lines 18-21, as they are necessary for Ray:
Note: leaving the 18-21 commented lead to this error:
ValueError: No default configuration for obs shape [21, 21, 1], you must specify conv_filters manually as a model option. Default configurations are only available for inputs of shape [42, 42, K] and [84, 84, K]. You may alternatively want to use a custom model or preprocessor.
sorry to interrupt you, when I input $ 'python trainners/runner.py -c trainners/trainnerV2.json -r Level-1' in Ubuntu, it always come out the error
2023-10-19 16:18:19,687 WARNING utils.py:569 -- Detecting docker specified CPUs. In previous versions of Ray, CPU detection in containers was incorrect. Please ensure that Ray has enough CPUs allocated. As a temporary workaround to revert to the prior behavior, set `RAY_USE_MULTIPROCESSING_CPU_COUNT=1` as an env var before starting Ray. Set the env var: `RAY_DISABLE_DOCKER_CPU_WARNING=1` to mute this warning.
2023-10-19 16:18:20,223 INFO services.py:1090 -- View the Ray dashboard at http://127.0.0.1:8265
2023-10-19 16:18:21,264 INFO trainer.py:617 -- Current log_level is WARN. For more information, set 'log_level': 'INFO' / 'DEBUG' or use the -v and -vv flags.
2023-10-19 16:18:26,004 WARNING util.py:40 -- Install gputil for GPU system monitoring.
Starting trainning without a priori knowledge
(pid=8395)
(pid=8395) /usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/rllib/utils/torch_ops.py:65: UserWarning: The given NumPy array is not writeable, and PyTorch does not support non-writeable tensors. This means you can write to the underlying (supposedly non-writeable) NumPy array using the tensor. You may want to copy the array to protect its data or make it writeable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /opt/conda/conda-bld/pytorch_1603729062494/work/torch/csrc/utils/tensor_numpy.cpp:141.)
(pid=8395) tensor = torch.from_numpy(np.asarray(item))
(pid=8397)
(pid=8397) /usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/rllib/utils/torch_ops.py:65: UserWarning: The given NumPy array is not writeable, and PyTorch does not support non-writeable tensors. This means you can write to the underlying (supposedly non-writeable) NumPy array using the tensor. You may want to copy the array to protect its data or make it writeable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /opt/conda/conda-bld/pytorch_1603729062494/work/torch/csrc/utils/tensor_numpy.cpp:141.)
(pid=8397) tensor = torch.from_numpy(np.asarray(item))
(pid=8394)
(pid=8394) /usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/rllib/utils/torch_ops.py:65: UserWarning: The given NumPy array is not writeable, and PyTorch does not support non-writeable tensors. This means you can write to the underlying (supposedly non-writeable) NumPy array using the tensor. You may want to copy the array to protect its data or make it writeable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /opt/conda/conda-bld/pytorch_1603729062494/work/torch/csrc/utils/tensor_numpy.cpp:141.)
(pid=8394) tensor = torch.from_numpy(np.asarray(item))
(pid=8400)
(pid=8400) /usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/rllib/utils/torch_ops.py:65: UserWarning: The given NumPy array is not writeable, and PyTorch does not support non-writeable tensors. This means you can write to the underlying (supposedly non-writeable) NumPy array using the tensor. You may want to copy the array to protect its data or make it writeable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /opt/conda/conda-bld/pytorch_1603729062494/work/torch/csrc/utils/tensor_numpy.cpp:141.)
(pid=8400) tensor = torch.from_numpy(np.asarray(item))
Traceback (most recent call last):
File "trainners/runner.py", line 81, in <module>
result = trainner.train()
File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/rllib/agents/trainer.py", line 517, in train
raise e
File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/rllib/agents/trainer.py", line 506, in train
result = Trainable.train(self)
File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/tune/trainable.py", line 336, in train
result = self.step()
File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/rllib/agents/trainer_template.py", line 147, in step
res = next(self.train_exec_impl)
File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/util/iter.py", line 756, in __next__
return next(self.built_iterator)
File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/util/iter.py", line 783, in apply_foreach
for item in it:
File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/util/iter.py", line 783, in apply_foreach
for item in it:
File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/util/iter.py", line 843, in apply_filter
for item in it:
File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/util/iter.py", line 843, in apply_filter
for item in it:
File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/util/iter.py", line 783, in apply_foreach
for item in it:
File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/util/iter.py", line 783, in apply_foreach
for item in it:
File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/util/iter.py", line 783, in apply_foreach
for item in it:
[Previous line repeated 1 more time]
File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/util/iter.py", line 876, in apply_flatten
for item in it:
File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/util/iter.py", line 828, in add_wait_hooks
item = next(it)
File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/util/iter.py", line 783, in apply_foreach
for item in it:
File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/util/iter.py", line 783, in apply_foreach
for item in it:
File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/util/iter.py", line 783, in apply_foreach
for item in it:
[Previous line repeated 1 more time]
File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/util/iter.py", line 471, in base_iterator
yield ray.get(futures, timeout=timeout)
File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/worker.py", line 1452, in get
raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(ValueError): ray::RolloutWorker.par_iter_next() (pid=8397, ip=172.17.0.7)
File "python/ray/_raylet.pyx", line 482, in ray._raylet.execute_task
File "python/ray/_raylet.pyx", line 436, in ray._raylet.execute_task.function_executor
File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/util/iter.py", line 1152, in par_iter_next
return next(self.local_it)
File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/rllib/evaluation/rollout_worker.py", line 317, in gen_rollouts
yield self.sample()
File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/rllib/evaluation/rollout_worker.py", line 621, in sample
batches = [self.input_reader.next()]
File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/rllib/evaluation/sampler.py", line 94, in next
batches = [self.get_data()]
File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/rllib/evaluation/sampler.py", line 211, in get_data
item = next(self.rollout_provider)
File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/rllib/evaluation/sampler.py", line 564, in _env_runner
_process_observations_w_trajectory_view_api(
File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/rllib/evaluation/sampler.py", line 1016, in _process_observations_w_trajectory_view_api
prep_obs: EnvObsType = _get_or_raise(preprocessors,
File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/rllib/models/preprocessors.py", line 166, in transform
self.check_shape(observation)
File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/rllib/models/preprocessors.py", line 62, in check_shape
raise ValueError(
ValueError: ('Observation ({}) outside given space ({})!', array([[[0.6],
is this the problem of versions of Ray and/or Gym? or could anyone provide a correct version of all the Lib we need?
I had to specify the paths at command line with some additional passed arguments. This is what I used (using GeneralExplorationPolicy as base directory):
$ python trainners/runner.py -c trainners/trainnerV2.json -r Level-1Additionally, in theutils.pyfile, I had to uncomment lines 18-21, as they are necessary for Ray:Note: leaving the 18-21 commented lead to this error:
ValueError: No default configuration for obs shape [21, 21, 1], you must specify conv_filters manually as a model option. Default configurations are only available for inputs of shape [42, 42, K] and [84, 84, K]. You may alternatively want to use a custom model or preprocessor.Originally posted by @t-woodw in https://github.com/dimikout3/MarsExplorer/issues/5#issuecomment-1289621252
sorry to interrupt you, when I input $ 'python trainners/runner.py -c trainners/trainnerV2.json -r Level-1' in Ubuntu, it always come out the error
is this the problem of versions of Ray and/or Gym? or could anyone provide a correct version of all the Lib we need?