int-brain-lab / mtscomp

Multichannel time series lossless compression in pure Python based on NumPy and zlib
BSD 3-Clause "New" or "Revised" License
35 stars 8 forks source link

Fix bug when using uint types for indexing #16

Closed mwshinn closed 3 months ago

mwshinn commented 3 months ago

Description:

When using uints as indexes on Reader objects, there is a crash.

arr = mtscomp.decompress(...)
arr[np.uint64(1):np.uint64(5)]

Error output:

File "/opt/miniconda3/envs/ks4/lib/python3.9/site-packages/mtscomp.py", line 832, in __getitem__
    out = arr[a:b:item.step, :]
TypeError: slice indices must be integers or None or have an __index__ method

Fix description:

This is caused by numpy promotion rules which specify that a uint minus an int is a float (see line 828 of mtscomp.py). This fixes the issue by casting all int indices to numpy ints, which will never cause an overflow and will always promote to int.

Code includes a fix and new test cases.

mwshinn commented 3 months ago

I should add as well that the reason for this fix is because Kilosort4 does not natively support mtscomp, but you can pass it in under the file_object argument. However, when passed this way, Kilosort4 sometimes gives uint indices.

data = mtscomp.decompress(...)
results = kilosort.run_kilosort(..., file_object=data)

This fix also fixed the Kilosort4 issue.