tonyzhaozh / aloha

MIT License
1.4k stars 242 forks source link

h5py_cache deprecated, expects old version of numpy #4

Closed DuaneNielsen closed 1 year ago

DuaneNielsen commented 1 year ago

on recording.py, got this exception...

Avg freq: 48.48 Get action: 0.000 Step env: 0.021
Traceback (most recent call last):
  File "record_episodes.py", line 230, in <module>
    main(vars(parser.parse_args()))
  File "record_episodes.py", line 190, in main
    is_healthy = capture_one_episode(DT, max_timesteps, camera_names, dataset_dir, dataset_name, overwrite)
  File "record_episodes.py", line 153, in capture_one_episode
    with h5py_cache.File(dataset_path + '.hdf5', 'w', chunk_cache_mem_size=1024**2*2) as root:
  File "/home/duane/miniconda3/envs/aloha/lib/python3.8/site-packages/h5py_cache/__init__.py", line 67, in File
    bytes_per_object = np.dtype(np.float).itemsize  # assume float as most likely
  File "/home/duane/.local/lib/python3.8/site-packages/numpy/__init__.py", line 305, in __getattr__
    raise AttributeError(__former_attrs__[attr])
AttributeError: module 'numpy' has no attribute 'float'.
`np.float` was a deprecated alias for the builtin `float`. To avoid this error in existing code, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.
The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:
    https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations

resolved by reverting to the commented code and adding rdcc_nbytes argument

from

    # import h5py_cache
    with h5py_cache.File(dataset_path + '.hdf5', 'w', chunk_cache_mem_size=1024**2*2) as root:
    # with h5py.File(dataset_path + '.hdf5', 'w') as root:

to

    # import h5py_cache
    # with h5py_cache.File(dataset_path + '.hdf5', 'w', chunk_cache_mem_size=1024**2*2) as root:
    with h5py.File(dataset_path + '.hdf5', 'w', rdcc_nbytes=1024**2*2) as root:
tonyzhaozh commented 1 year ago

Thanks for catching it Duane!

Indeed the line you suggested works well. I had tried to use the rdcc_nbytes flag before, but for some reason it did not work (it saves things but really slowly.) I tested the lines you suggested and the saving speed is the same between h5py_cache and h5py. I just pushed these changes to main see this commit.

Also you are more than welcome to open a pull request for issues like this, so you will be properly credited in the commit history!

Aloha. Tony