search

CINPLA / expipe-plugin-cinpla

Plugins for expipe command line interface
http://expipe-plugin-cinpla.readthedocs.io/en/latest/
GNU General Public License v3.0
1 stars 3 forks source link

error when spikesorting KS2 #68

Closed mariapfj closed 3 months ago

mariapfj commented 4 months ago

running into an error when attempting to spikesort with kilosort2 (on action registered directly in educloud): (also it registered a few bad channels; ['CH36' 'CH57' 'CH61' 'CH62' 'CH64'] but this is using the 32ch json probe file you provided, and as far as I've been able to tell by loading the file it definitely shouldn't go past 31?)

####################################################################################

ERROR: unable to process 158-290224-1 Traceback (most recent call last): File "/fp/projects01/ec109/software/expipe-plugin-cinpla/src/expipe_plugin_cinpla/scripts/process.py", line 218, in process_ecephys sorting = ss.run_sorter( ^^^^^^^^^^^^^^ File "/fp/homes01/u01/ec-mariapfj/.local/lib/python3.11/site-packages/spikeinterface/sorters/runsorter.py", line 174, in run_sorter return run_sorter_local(**common_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/fp/homes01/u01/ec-mariapfj/.local/lib/python3.11/site-packages/spikeinterface/sorters/runsorter.py", line 224, in run_sorter_local SorterClass.run_from_folder(output_folder, raise_error, verbose) File "/fp/homes01/u01/ec-mariapfj/.local/lib/python3.11/site-packages/spikeinterface/sorters/basesorter.py", line 293, in run_from_folder raise SpikeSortingError( spikeinterface.sorters.utils.misc.SpikeSortingError: Spike sorting error trace: Traceback (most recent call last): File "/fp/homes01/u01/ec-mariapfj/.local/lib/python3.11/site-packages/spikeinterface/sorters/basesorter.py", line 258, in run_from_folder SorterClass._run_from_folder(sorter_output_folder, sorter_params, verbose) File "/fp/homes01/u01/ec-mariapfj/.local/lib/python3.11/site-packages/spikeinterface/sorters/external/kilosortbase.py", line 217, in _run_from_folder raise Exception(f"{cls.sorter_name} returned a non-zero exit code") Exception: kilosort2 returned a non-zero exit code

Spike sorting failed. You can inspect the runtime trace in /projects/ec109/maria/novel-env-nwb/actions/158-290224-1/data/spikeinterface/kilosort2/spikesorting/spikeinterface_log.json.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/fp/projects01/ec109/software/expipe-plugin-cinpla/src/expipe_plugin_cinpla/widgets/process.py", line 296, in on_run process.process_ecephys( File "/fp/projects01/ec109/software/expipe-plugin-cinpla/src/expipe_plugin_cinpla/scripts/process.py", line 234, in process_ecephys raise Exception(f"Spike sorting failed:\n\n{e}") Exception: Spike sorting failed:

Spike sorting error trace: Traceback (most recent call last): File "/fp/homes01/u01/ec-mariapfj/.local/lib/python3.11/site-packages/spikeinterface/sorters/basesorter.py", line 258, in run_from_folder SorterClass._run_from_folder(sorter_output_folder, sorter_params, verbose) File "/fp/homes01/u01/ec-mariapfj/.local/lib/python3.11/site-packages/spikeinterface/sorters/external/kilosortbase.py", line 217, in _run_from_folder raise Exception(f"{cls.sorter_name} returned a non-zero exit code") Exception: kilosort2 returned a non-zero exit code

Spike sorting failed. You can inspect the runtime trace in /projects/ec109/maria/novel-env-nwb/actions/158-290224-1/data/spikeinterface/kilosort2/spikesorting/spikeinterface_log.json.

alejoe91 commented 4 months ago

an you check the log file?

mariapfj commented 4 months ago

I'm having some issues locating the log file, there's nothing in the data folder except main.nwb and from what I've been able to inspect I can't find any traces of spikeinterface?

mariapfj commented 4 months ago

so the directory is created when I start the processing, but the second it fails the directory is deleted and I'm kicked out. not sure how to help you here

alejoe91 commented 4 months ago

Ah probably it gets cleaned up.. does it happen with all actions? Or were you able to run it some times?

mariapfj commented 4 months ago

so far its failed for 3 different recs, all from the same experiment and all registered directly in nwb (rather than converted from exdir)

alejoe91 commented 4 months ago

Are you unchecking the sort by group??

mariapfj commented 4 months ago

yes

alejoe91 commented 4 months ago

I'll take a look

alejoe91 commented 4 months ago

Maria, I think it could be a problem with MATLAB. Let's discuss on slack

alejoe91 commented 3 months ago

turns out that this was an attempt to run KS on a non-GPU machine :)