BUNPC / Homer3

MATLAB application for fNIRS data processing and visualization
91 stars 54 forks source link

[BUG]: Error in hdf5write_safe>write_integer in 64-bit Linux/Unix systems due to HDF5 datatype change in 1.78.0 #181

Open dim-ask opened 1 year ago

dim-ask commented 1 year ago

Version of Homer3 you are using

Latest master branch (1.80.2), tested also on latest development branch 1.80.4. The error appears to be here since v1.78.0.

Your environment (MATLAB environment and OS)

Errors on ubuntu, macos and (Matlab 2022b, and 2023b prerelease). Not errors on Windows 11 (Matlab 2023a).

Description of the issue

Problem lies on the line

https://github.com/BUNPC/Homer3/blob/666ab6d0b054e9b2eb829d62736aaa2148638f67/DataTree/AcquiredData/DataFiles/Hdf5/hdf5write_safe.m#L98

as the datatype H5T_NATIVE_ULONG differs among different systems [1], and in particular in 64-bit linux and macos it expresses 64-bit unsigned integers, while in Windows and 32-bit linux 32-bit unsigned integers. Thus, in linux and macos, I get an error in 3 lines after, on

https://github.com/BUNPC/Homer3/blob/666ab6d0b054e9b2eb829d62736aaa2148638f67/DataTree/AcquiredData/DataFiles/Hdf5/hdf5write_safe.m#L101

as it essentially tries to write an int32 into a H5T_STD_U64LE. In windows I get no error.

This is due to this commit that changed H5T_NATIVE_INT into H5T_NATIVE_ULONG.

Steps to reproduce:

Just try to write any .snirf file to disk in 64-bit linux/macos. (eg run Nirs2Snirf inside Homer3/SubjDataSample).

Steps to fix:

Consider reverting H5T_NATIVE_ULONG back to H5T_NATIVE_INT or any other compatible platform indepedent type. Not sure if there were performance issues with that, I checked also H5T_NATIVE_UINT32 or H5T_NATIVE_INT32 and they work.

Actual behavior:

The error message


>> Nirs2Snirf

Converting /home/xxx/Documents/MATLAB/Homer3/SubjDataSample/test.nirs to /home/xxx/Documents/MATLAB/Homer3/SubjDataSample/test.snirf
Error using hdf5lib2
The class of input data must be integer instead of int32 when the HDF5 class is H5T_STD_U64LE.

Error in H5D.write (line 100)
H5ML.hdf5lib2('H5Dwrite', varargin{:});

Error in hdf5write_safe>write_integer (line 101)
H5D.write(dsid, tid, 'H5S_ALL', 'H5S_ALL', 'H5P_DEFAULT', int32(val));

Error in hdf5write_safe (line 52)
        write_integer(fid, name, val);

Error in MeasListClass/SaveHdf5 (line 171)
            hdf5write_safe(fid, [location, '/sourceIndex'], uint64(obj.sourceIndex));

Error in DataClass/SaveHdf5 (line 248)
                obj.measurementList(ii).SaveHdf5(fid, [location, '/measurementList', num2str(ii)]);

Error in SnirfClass/SaveData (line 672)
                obj.data(ii).SaveHdf5(fileobj, [obj.location, '/data', num2str(ii)]);

Error in SnirfClass/SaveHdf5 (line 742)
                obj.SaveData(obj.fid);

Error in FileLoadSaveClass/Save (line 78)
                        obj.SaveHdf5(filename, params);

Error in Nirs2Snirf (line 65)
    snirf(ii).Save(dst);

[1] https://www.ibm.com/docs/en/ibm-mq/9.1?topic=platforms-standard-data-types-unix-linux-windows

LucyRah commented 1 year ago

Thanks so much for pointing this out! I have the same issue and solved it by replacing the H5T_NATIVE_ULONG back to H5T_NATIVE_INT