search

wtclarke / pymapvbvd

Python port of mapVBVD
MIT License
40 stars 25 forks source link

Reading fails on Windows for files >2GB #38

Closed FrankZijlstra closed 11 months ago

FrankZijlstra commented 1 year ago

In twix_map_obj.readData, mem is cast to int, which on Windows defaults to int32. This fails when data indices are larger than 2**31 - 1, i.e. on files larger than 2GB. The cast should be explicitly specify np.int64.

wtclarke commented 11 months ago

Hi @FrankZijlstra , Sorry for the delay in getting around to this. Could you see if the fix that you recommended, implemented in the bf/largefileswin branch works?

FrankZijlstra commented 11 months ago

Seems to work fine!

While I have your attention, there is another small, very rare bug. On line https://github.com/wtclarke/pymapvbvd/blob/fb188f05c90c8ad4b074dca413d57abcfcb157bf/mapvbvd/mapVBVD.py#L163 the check for n_acq == len(filePos) should not be there (it's not there in the MATLAB mapvbvd), as it will drop the last readout when the size of filePos is exactly a multiple of allocSize. Instead, on line https://github.com/wtclarke/pymapvbvd/blob/fb188f05c90c8ad4b074dca413d57abcfcb157bf/mapvbvd/mapVBVD.py#L143 change the > to a >= (i.e. reserve space for the last readout).

wtclarke commented 11 months ago

Great, both these issues should be fixed by PR #41 and will be in version 0.5.6.

wtclarke commented 11 months ago

Thanks for taking the time to hunt these subtle bugs down and report them!