search

telegraphic / hickle

a HDF5-based python pickle replacement
http://telegraphic.github.io/hickle/
Other
485 stars 70 forks source link

hickle 3 legacy loader and h5py 3.x string issue #158

Closed hernot closed 2 years ago

hernot commented 2 years ago

Just encountered that the hickle 3.x legacy loader may not properly load back strings when h5py 3.x is installed instead of h5py 2.10. The issue is that h5py has changed the way string datasets are interpreted. Under the string

'incartdb' or the string 'I05' are returned on load properly. With h5py >= 3.0 installed both strings appear as "b'incartdb'" and "b'I05'". As I do not have sufficient knowledge about legacy loader which uses completely different approach and concept compared to hickle 4 and hickle 5 I do not feel the appropriate person to figure how to mitigate dependent upon h5py version.

The issue does not appear in hickle 5 and i guess also hickle 4 as sufficient provisions are taken there.

telegraphic commented 2 years ago

As this is not an issue in v5, I vote to not fix this. So I'm closing this issue -- but if someone wants to tackle this happy for it to be reopened.

hernot commented 2 years ago

I guess a note in README or manual that H5PY 2.10 has to be installed manually first before installing hickle should be sufficient as a fix for loading and converting old data to new file formats with hickle using recent h5py versions.