large .mat files when data is loaded from .mat file

[ivosonntag]

Hi Jinghao, I started running min1pipe from previously created .mat files containing a single pre-concatanated dataset. The problem is that when min1pipe saves the dataset through data_cat to get the xxx_frame_all.mat file, the file is roughly 3.7 times as large as the original dataset (16bit), even though it should only be 2 times as large (single). This does not seem to be a problem for the further processing (I will update on that), however I fail to understand how this can happen. The only variable created in the new .mat file is exactly the size it should be and single-precision.

Also when I run min1pipe on a .mat file that was previously created by min1pipe the size is not changed at all.....

Do you have any idea what might cause this?

[JinghaoLu]

Hi Ivo,

Thanks for bringing this out. Actually I used h5 functions to write data to uncompressed mat file for speed, and thus sacrifice the storage size. Since mat file v7.3 uses hdf5 format which has a huge storage overhead, the current frame_all.mat and reg.mat are all larger than the variable itself unless you make spatial downsampling if you find the storage consumption is too much. Better storage solutions with optimal read&write speed are also welcome.

Thanks,

Jinghao