SpikeInterface / spikeinterface

A Python-based module for creating flexible and robust spike sorting pipelines.
https://spikeinterface.readthedocs.io
MIT License
531 stars 188 forks source link

CompressedBinaryIblExtractor does not save cbin_file relative #3523

Closed arturoptophys closed 2 days ago

arturoptophys commented 2 weeks ago

Hi all,

i notices when dumping a CompressedBinaryIblExtractor to json the path to cbin_file is not saved relative but absolute, which leads to problems when processing it elsewhere.

"class": "spikeinterface.extractors.cbin_ibl.CompressedBinaryIblExtractor",
"module": "spikeinterface",
"version": "0.101.2",
"kwargs": {
    "folder_path": "pathremoved/20241105_r0118_wt_1456/ephys",
    "load_sync_channel": false,
    "cbin_file": "/mnt/pathremoved/20241105_r0118_wt_1456/ephys/20241105_r0118_wt_1456.imec0.ap.cbin"
},
...
 "relative_paths": true

I found the culprit : https://github.com/SpikeInterface/spikeinterface/blob/e525d85948009e7dc57579a8a428502a5534c111/src/spikeinterface/core/core_tools.py#L338 assumes paths should have "path" in the name to be made relative. This should also include "cbin_file" or maybe just "file".

Best, Artur

arturoptophys commented 2 weeks ago

Related to that.. how does the Sorting Analyzer saves the recording provenance ? I tried to "fix" the path issue by modifiyng the cbin_file to the absolute path on the computing cluster. Spikesorting run with no problems and sorting analyzer was created and written to zarr. Now if i load this analyzer on a different computer it does not have a recording object associated with it and i cant do things like analyzer.sorting.get_total_duration()

alejoe91 commented 2 weeks ago

Hi, see this PR with a fix: #3524

As long as the relative path between the sorting analyzer folder and the recording is unchanged, the recording will be loaded. If the relative path changes, then the sorting analyzer will not be able to find the recording.

What you can do in this case is to load the sorting analyzer without the recording, load the recording "externally", and set it temporarily to the sorting analyzer with the analyzer.set_temporary_recording() function

arturoptophys commented 2 weeks ago

Yes, https://github.com/SpikeInterface/spikeinterface/pull/3524 fixes the issue. At least the sorting runs now without path modifications, sorting analyzer will check once sorting is done.

To note: analyzer.set_temporary_recording() registers recording for the analyzer but not for sorting. After it i can do analyzer.recording.get_duration() but not analyzer.sorting.get_total_duration().

alejoe91 commented 2 weeks ago

you can directly do analyzer.get_total_duration(). I think in general this is safer because the duration is actually a property of the recording, not the sorting