frejanordsiek / hdf5storage

Python package to read and write a wide range of Python types to/from HDF5 formatted files. Can read/write data to the HDF5 based Matlab v7.3 MAT files.
BSD 2-Clause "Simplified" License
83 stars 24 forks source link

v0.1.18 loadmat() not loading python float/ints from saved mat-file under NumPy 1.23 #122

Closed jeremypriest closed 1 year ago

jeremypriest commented 2 years ago

Note: I wrote all of the below out, and then realized that it appears this issue has been fixed on the main branch. A release of the main branch would be much appreciated.

hdf5storage 0.1.18 is affected by some sort of change in NumPy 1.23. If a python float or int is saved out, it will not be read back in during loadmat(). When it is saved out as a np.float32/64 or np.int32/64, the variable is loaded in properly.

Here is the code example I'm using:

from hdf5storage import loadmat, savemat
import numpy as np

float1 = float(1)
float2 = np.float64(1)

file_mat = 'test.mat'
out_mat = {'float': float1,
           'np_float64': float2}
print(out_mat.keys())

savemat(file_mat, out_mat)

in_mat = loadmat(file_mat)
print(in_mat.keys())

print(np.__version__)

Output under NumPy 1.23.3:

dict_keys(['float', 'np_float64'])
dict_keys(['np_float64'])
1.23.3

Output under NumPy 1.22.4:

dict_keys(['float', 'np_float64'])
dict_keys(['float', 'np_float64'])
1.22.4

Output under NumPy 1.23.3 and hdf5storage main branch (commit ec2e1e34789ad27713b860eb4de5da99817fa315):

dict_keys(['float', 'np_float64'])
dict_keys(['float', 'np_float64'])
1.23.3
frejanordsiek commented 1 year ago

This is fixed in the a06c749b970d012256311ba0b95321ac83cae0ec commit on the 0.1.x branch. It is caused by the numpy.asscalar function being deprecated, which prevented reading the value (it was saved, though). I will be pushing a 0.1.19 bugfix release to PyPI shortly.