HDFGroup / h5pyd

h5py distributed - Python client library for HDF Rest API
Other
110 stars 39 forks source link

Replace get_chunk_info with chunk_iter #148

Closed mattjala closed 1 year ago

mattjala commented 1 year ago

get_chunk_info internally has redundant iteration, so this should result in significant speedup in some cases

Not all versions of the HDF5 library have chunk_iter, so this will fallback to the original behavior in certain library versions.

Testing with satellite data showed a speedup of about 10% (2m54s -> 2m33s)