I'm currently using Ray for RL. The observation space in my project is a dict which contain a long MultiDiscrete space(length=52). The error occurred at the point that Ray framework try to convert the space to tensor(I believe this is not a torch bug, because tensorflow backend meet the same problem).
It seems that the code try to create numpy.ndarray with raw data using ndarray(data), but the correct way should be numpy.array(data). But array solution would cause another problem that the nested data type will lose. In my situation, data is a list of tensor, but numpy.array(data) return a common ndarray.
Environment:
Archlinux x86_64, Install all package with mamba(conda-forge) package:
pytorch 2.1.0
Ray 2.7.1
tree 0.1.8
StackTrace:
File "/home/shilka/miniforge3/envs/ray/lib/python3.9/site-packages/ray/rllib/utils/torch_utils.py", line 262, in convert_to_torch_tensor
return tree.map_structure(mapping, x)
File "/home/shilka/miniforge3/envs/ray/lib/python3.9/site-packages/tree/init.py", line 434, in map_structure
return unflatten_as(structures[0],
File "/home/shilka/miniforge3/envs/ray/lib/python3.9/site-packages/tree/init.py", line 380, in unflattenas
, packed = _packed_nest_with_indices(structure, flat_sequence, 0)
File "/home/shilka/miniforge3/envs/ray/lib/python3.9/site-packages/tree/init.py", line 314, in _packed_nest_with_indices
for s in _yield_value(structure):
File "/home/shilka/miniforge3/envs/ray/lib/python3.9/site-packages/tree/init.py", line 132, in _yieldvalue
for , v in _yield_sorted_items(iterable):
File "/home/shilka/miniforge3/envs/ray/lib/python3.9/site-packages/tree/init.py", line 158, in _yield_sorted_items
yield key, iterable[key]
File "/home/shilka/miniforge3/envs/ray/lib/python3.9/site-packages/ray/rllib/policy/sample_batch.py", line 932, in getitem
self.intercepted_values[key] = self.get_interceptor(value)
File "/home/shilka/miniforge3/envs/ray/lib/python3.9/site-packages/ray/rllib/utils/torch_utils.py", line 262, in convert_to_torch_tensor
return tree.map_structure(mapping, x)
File "/home/shilka/miniforge3/envs/ray/lib/python3.9/site-packages/tree/init.py", line 434, in map_structure
return unflatten_as(structures[0],
File "/home/shilka/miniforge3/envs/ray/lib/python3.9/site-packages/tree/init.py", line 380, in unflattenas
, packed = _packed_nest_with_indices(structure, flat_sequence, 0)
File "/home/shilka/miniforge3/envs/ray/lib/python3.9/site-packages/tree/init.py", line 317, in _packed_nest_with_indices
packed.append(_sequence_like(s, child))
File "/home/shilka/miniforge3/envs/ray/lib/python3.9/site-packages/tree/sequence.py", line 111, in _sequence_like
return type(instance)(args)
ValueError: maximum supported dimension for an ndarray is 32, found 52
Values on final call raising exception:
both instance and args length is 52, they are numpy.ndarray and list object:
instance: array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0.], dtype=float32)
args: [tensor(0.), tensor(0.), tensor(0.), tensor(0.), tensor(0.), tensor(0.), tensor(0.), tensor(0.), tensor(0.), tensor(0.), tensor(0.), tensor(0.), tensor(0.), tensor(0.), ...]
I'm currently using Ray for RL. The observation space in my project is a dict which contain a long MultiDiscrete space(length=52). The error occurred at the point that Ray framework try to convert the space to tensor(I believe this is not a torch bug, because tensorflow backend meet the same problem). It seems that the code try to create numpy.ndarray with raw data using ndarray(data), but the correct way should be numpy.array(data). But array solution would cause another problem that the nested data type will lose. In my situation, data is a list of tensor, but numpy.array(data) return a common ndarray.
Environment: Archlinux x86_64, Install all package with mamba(conda-forge) package: pytorch 2.1.0 Ray 2.7.1 tree 0.1.8
StackTrace: File "/home/shilka/miniforge3/envs/ray/lib/python3.9/site-packages/ray/rllib/utils/torch_utils.py", line 262, in convert_to_torch_tensor return tree.map_structure(mapping, x) File "/home/shilka/miniforge3/envs/ray/lib/python3.9/site-packages/tree/init.py", line 434, in map_structure return unflatten_as(structures[0], File "/home/shilka/miniforge3/envs/ray/lib/python3.9/site-packages/tree/init.py", line 380, in unflattenas , packed = _packed_nest_with_indices(structure, flat_sequence, 0) File "/home/shilka/miniforge3/envs/ray/lib/python3.9/site-packages/tree/init.py", line 314, in _packed_nest_with_indices for s in _yield_value(structure): File "/home/shilka/miniforge3/envs/ray/lib/python3.9/site-packages/tree/init.py", line 132, in _yieldvalue for , v in _yield_sorted_items(iterable): File "/home/shilka/miniforge3/envs/ray/lib/python3.9/site-packages/tree/init.py", line 158, in _yield_sorted_items yield key, iterable[key] File "/home/shilka/miniforge3/envs/ray/lib/python3.9/site-packages/ray/rllib/policy/sample_batch.py", line 932, in getitem self.intercepted_values[key] = self.get_interceptor(value) File "/home/shilka/miniforge3/envs/ray/lib/python3.9/site-packages/ray/rllib/utils/torch_utils.py", line 262, in convert_to_torch_tensor return tree.map_structure(mapping, x) File "/home/shilka/miniforge3/envs/ray/lib/python3.9/site-packages/tree/init.py", line 434, in map_structure return unflatten_as(structures[0], File "/home/shilka/miniforge3/envs/ray/lib/python3.9/site-packages/tree/init.py", line 380, in unflattenas , packed = _packed_nest_with_indices(structure, flat_sequence, 0) File "/home/shilka/miniforge3/envs/ray/lib/python3.9/site-packages/tree/init.py", line 317, in _packed_nest_with_indices packed.append(_sequence_like(s, child)) File "/home/shilka/miniforge3/envs/ray/lib/python3.9/site-packages/tree/sequence.py", line 111, in _sequence_like return type(instance)(args) ValueError: maximum supported dimension for an ndarray is 32, found 52
Values on final call raising exception: both instance and args length is 52, they are numpy.ndarray and list object: instance: array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], dtype=float32) args: [tensor(0.), tensor(0.), tensor(0.), tensor(0.), tensor(0.), tensor(0.), tensor(0.), tensor(0.), tensor(0.), tensor(0.), tensor(0.), tensor(0.), tensor(0.), tensor(0.), ...]