search

SpikeInterface / spikeinterface

A Python-based module for creating flexible and robust spike sorting pipelines.
https://spikeinterface.readthedocs.io
MIT License
527 stars 187 forks source link

spikesorting failed kilosort4 (ValueError: n_samples=2 should be >= n_clusters=6.) #2819

Closed chiyu1203 closed 6 months ago

chiyu1203 commented 6 months ago

I had some good luck with kilosort 4 on Spikeinterface last month but when I repeated the same analysis this month, spike sorting kept failing. My kilosort is 4.0.4 and my spikeinterface is 100.4 (have tried 100.6 as well but stilll saw the error) and my python version is 3.10.14 (have tried python 3.9 and 3.11, but saw the same error) Below is the command I used to call kilosort 4. Could anyone suggest me what might be the issue?

import spikeinterface.sorters as ss
import spikeinterface.core as si
recording_saved = si.read_zarr(oe_folder / "preprocessed_compressed.zarr")
sorter_params={'dminx': 250,'nearest_templates':10}
sorting_spikes = ss.run_sorter(sorter_name=this_sorter, recording=recording_saved, remove_existing_folder=True, output_folder=oe_folder / result_folder_name, verbose=True, **sorter_params)

And this is the error I saw

Exception has occurred: SpikeSortingError Spike sorting error trace: Traceback (most recent call last): File "c:\Users\einat\anaconda3\envs\ephys\lib\site-packages\spikeinterface\sorters\basesorter.py", line 258, in run_from_folder SorterClass._run_from_folder(sorter_output_folder, sorter_params, verbose) File "c:\Users\einat\anaconda3\envs\ephys\lib\site-packages\spikeinterface\sorters\external\kilosort4.py", line 235, in _run_from_folder ops, bfile, st0 = compute_drift_correction( File "c:\Users\einat\anaconda3\envs\ephys\lib\site-packages\kilosort\run_kilosort.py", line 343, in compute_drift_correction ops, st = datashift.run(ops, bfile, device=device, progress_bar=progressbar) File "c:\Users\einat\anaconda3\envs\ephys\lib\site-packages\kilosort\datashift.py", line 192, in run st, , ops = spikedetect.run(ops, bfile, device=device, progress_bar=progress_bar) File "c:\Users\einat\anaconda3\envs\ephys\lib\site-packages\kilosort\spikedetect.py", line 188, in run ops['wPCA'], ops['wTEMP'] = extract_wPCA_wTEMP( File "c:\Users\einat\anaconda3\envs\ephys\lib\site-packages\kilosort\spikedetect.py", line 74, in extract_wPCA_wTEMP model = KMeans(n_clusters=ops['settings']['n_templates'], n_init = 10).fit(clips) File "c:\Users\einat\anaconda3\envs\ephys\lib\site-packages\sklearn\base.py", line 1474, in wrapper return fit_method(estimator, *args, **kwargs) File "c:\Users\einat\anaconda3\envs\ephys\lib\site-packages\sklearn\cluster_kmeans.py", line 1490, in fit self._check_params_vs_input(X) File "c:\Users\einat\anaconda3\envs\ephys\lib\site-packages\sklearn\cluster_kmeans.py", line 1431, in _check_params_vs_input super()._check_params_vs_input(X, default_n_init=10) File "c:\Users\einat\anaconda3\envs\ephys\lib\site-packages\sklearn\cluster_kmeans.py", line 879, in _check_params_vs_input raise ValueError( ValueError: n_samples=2 should be >= n_clusters=6.

Spike sorting failed. You can inspect the runtime trace in C:\Users\einat\Documents\Open Ephys\2024-05-07_23-08-55\results_KS4/spikeinterface_log.json. File "C:\Users\einat\Documents\GitHub\ephys\raw2si.py", line 162, in main sorting_spikes = ss.run_sorter(sorter_name=this_sorter, recording=recording_saved, remove_existing_folder=True, File "C:\Users\einat\Documents\GitHub\ephys\raw2si.py", line 286, in recording_saved,sorting_spikes=main(thisDir, json_file) spikeinterface.sorters.utils.misc.SpikeSortingError: Spike sorting error trace: Traceback (most recent call last): File "c:\Users\einat\anaconda3\envs\ephys\lib\site-packages\spikeinterface\sorters\basesorter.py", line 258, in run_from_folder SorterClass._run_from_folder(sorter_output_folder, sorter_params, verbose) File "c:\Users\einat\anaconda3\envs\ephys\lib\site-packages\spikeinterface\sorters\external\kilosort4.py", line 235, in _run_from_folder ops, bfile, st0 = compute_drift_correction( File "c:\Users\einat\anaconda3\envs\ephys\lib\site-packages\kilosort\run_kilosort.py", line 343, in compute_drift_correction ops, st = datashift.run(ops, bfile, device=device, progress_bar=progressbar) File "c:\Users\einat\anaconda3\envs\ephys\lib\site-packages\kilosort\datashift.py", line 192, in run st, , ops = spikedetect.run(ops, bfile, device=device, progress_bar=progress_bar) File "c:\Users\einat\anaconda3\envs\ephys\lib\site-packages\kilosort\spikedetect.py", line 188, in run ops['wPCA'], ops['wTEMP'] = extract_wPCA_wTEMP( File "c:\Users\einat\anaconda3\envs\ephys\lib\site-packages\kilosort\spikedetect.py", line 74, in extract_wPCA_wTEMP model = KMeans(n_clusters=ops['settings']['n_templates'], n_init = 10).fit(clips) File "c:\Users\einat\anaconda3\envs\ephys\lib\site-packages\sklearn\base.py", line 1474, in wrapper return fit_method(estimator, *args, **kwargs) File "c:\Users\einat\anaconda3\envs\ephys\lib\site-packages\sklearn\cluster_kmeans.py", line 1490, in fit self._check_params_vs_input(X) File "c:\Users\einat\anaconda3\envs\ephys\lib\site-packages\sklearn\cluster_kmeans.py", line 1431, in _check_params_vs_input super()._check_params_vs_input(X, default_n_init=10) File "c:\Users\einat\anaconda3\envs\ephys\lib\site-packages\sklearn\cluster_kmeans.py", line 879, in _check_params_vs_input raise ValueError( ValueError: n_samples=2 should be >= n_clusters=6.

And then this is the spikeinterface_log.jason file

{ "sorter_name": "kilosort4", "sorter_version": "4", "datetime": "2024-05-08T02:38:03.003746", "runtime_trace": [], "error": true, "error_trace": "Traceback (most recent call last):\n File \"c:\Users\einat\anaconda3\envs\ephys\lib\site-packages\spikeinterface\sorters\basesorter.py\", line 258, in run_from_folder\n SorterClass._run_from_folder(sorter_output_folder, sorter_params, verbose)\n File \"c:\Users\einat\anaconda3\envs\ephys\lib\site-packages\spikeinterface\sorters\external\kilosort4.py\", line 227, in _run_from_folder\n ops, bfile, st0 = compute_drift_correction(\n File \"c:\Users\einat\anaconda3\envs\ephys\lib\site-packages\kilosort\run_kilosort.py\", line 343, in compute_drift_correction\n ops, st = datashift.run(ops, bfile, device=device, progress_bar=progressbar)\n File \"c:\Users\einat\anaconda3\envs\ephys\lib\site-packages\kilosort\datashift.py\", line 192, in run\n st, , ops = spikedetect.run(ops, bfile, device=device, progress_bar=progress_bar)\n File \"c:\Users\einat\anaconda3\envs\ephys\lib\site-packages\kilosort\spikedetect.py\", line 188, in run\n ops['wPCA'], ops['wTEMP'] = extract_wPCA_wTEMP(\n File \"c:\Users\einat\anaconda3\envs\ephys\lib\site-packages\kilosort\spikedetect.py\", line 74, in extract_wPCA_wTEMP\n model = KMeans(n_clusters=ops['settings']['n_templates'], n_init = 10).fit(clips)\n File \"c:\Users\einat\anaconda3\envs\ephys\lib\site-packages\sklearn\base.py\", line 1474, in wrapper\n return fit_method(estimator, *args, **kwargs)\n File \"c:\Users\einat\anaconda3\envs\ephys\lib\site-packages\sklearn\cluster\_kmeans.py\", line 1490, in fit\n self._check_params_vs_input(X)\n File \"c:\Users\einat\anaconda3\envs\ephys\lib\site-packages\sklearn\cluster\_kmeans.py\", line 1431, in _check_params_vs_input\n super()._check_params_vs_input(X, default_n_init=10)\n File \"c:\Users\einat\anaconda3\envs\ephys\lib\site-packages\sklearn\cluster\_kmeans.py\", line 879, in _check_params_vs_input\n raise ValueError(\nValueError: n_samples=2 should be >= n_clusters=6.\n", "run_time": null }

chiyu1203 commented 6 months ago

My gut feeling is that that recording session I tested today, does not have any putative spikes...