search

int-brain-lab / mtscomp

Multichannel time series lossless compression in pure Python based on NumPy and zlib
BSD 3-Clause "New" or "Revised" License
33 stars 8 forks source link

Can not compress catgt files (tcat) #10

Open yonatanfa opened 2 years ago

yonatanfa commented 2 years ago

Hey, I've been using mtscomp for compressing ~200 files, with great efficiency. I compress Neuropixels data files (ap bin). For some reason, I cannot compress the next level of analysis (after processing with Catgt). These are tcat files, which have seemingly good .meta files. The error says that something is wrong with the meta file (example meta file attached). Do you have any insight in the topic? Thanks! Yonatan image

rossant commented 2 years ago

Could you post the command and the full error message?

yonatanfa commented 2 years ago

Thanks for the response. Command: from ibllib.io.spikeglx import Reader raw = Reader(ap_file) # Instantiate a splikeglx Reader object compressed = raw.compress_file(keep_original=True) # Compress the file and SAVE THE RAW SOURCE FILE

Error:

Traceback (most recent call last): File "C:\Users\owner\Anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 3418, in run_code exec(code_obj, self.user_global_ns, self.user_ns) File "", line 1, in runfile('D:/Python/e_phys/compression.py', wdir='D:/Python/e_phys') File "C:\Program Files\JetBrains\PyCharm 2019.1.3\helpers\pydev_pydev_bundle\pydev_umd.py", line 197, in runfile pydev_imports.execfile(filename, global_vars, local_vars) # execute the script File "C:\Program Files\JetBrains\PyCharm 2019.1.3\helpers\pydev_pydev_imps_pydev_execfile.py", line 18, in execfile exec(compile(contents+"\n", file, 'exec'), glob, loc) File "D:/Python/e_phys/compression.py", line 29, in raw = Reader(ap_file) # Instantiate a splikeglx Reader object File "C:\Users\owner\Anaconda3\lib\site-packages\ibllib\io\spikeglx.py", line 47, in init self.meta = read_meta_data(file_meta_data) File "C:\Users\owner\Anaconda3\lib\site-packages\ibllib\io\spikeglx.py", line 336, in read_meta_data k, v = a.split('=') ValueError: too many values to unpack (expected 2)

Error in screenshot image

Here's the metadata file (in a text format, not a .meta because I couldn't upload it): MetaData - Copy to txt.txt

rossant commented 2 years ago

This appears to be using the ibllib wrapper, could you post the issue there instead? As a standalone library, mtscomp doesn't directly support different file formats.