error with spyking circus2

[paul-aparicio]

Hello, I am running spikeinterface v0.101.2 and attempting to sort data from a 96 channel blackrock utah probe. Since the contacts on each probe are ~400um apart, I attempting to sort by grouping property (where each shank of the probe is a group):

# run the sorter
sorting_SC = si.run_sorter_by_property(sorter_name='spykingcircus2', recording=recording_cache, grouping_property='group', folder=base_folder+'processed/results_sc', **sorter_params)

It would appear that it is trying to whiten the data. I get the following error:

SpikeSortingError: Spike sorting error trace: Traceback (most recent call last): File "/home/paul/anaconda3/envs/sin/lib/python3.11/site-packages/spikeinterface/sorters/basesorter.py", line 261, in run_from_folder SorterClass._run_from_folder(sorter_output_folder, sorter_params, verbose) File "/home/paul/anaconda3/envs/sin/lib/python3.11/site-packages/spikeinterface/sorters/internal/spyking_circus2.py", line 152, in _run_from_folder recording_w = whiten(recording_f, mode="local", radius_um=radius_um, dtype="float32", regularize=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/paul/anaconda3/envs/sin/lib/python3.11/site-packages/spikeinterface/preprocessing/whiten.py", line 84, in init W, M = compute_whitening_matrix( ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/paul/anaconda3/envs/sin/lib/python3.11/site-packages/spikeinterface/preprocessing/whiten.py", line 209, in compute_whitening_matrix estimator.fit(data) File "/home/paul/anaconda3/envs/sin/lib/python3.11/site-packages/sklearn/base.py", line 1473, in wrapper return fit_method(estimator, *args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/paul/anaconda3/envs/sin/lib/python3.11/site-packages/sklearn/covariance/_graph_lasso.py", line 965, in fit X = self._validate_data(X, ensure_min_features=2) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/paul/anaconda3/envs/sin/lib/python3.11/site-packages/sklearn/base.py", line 633, in _validate_data out = check_array(X, input_name="X", check_params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/paul/anaconda3/envs/sin/lib/python3.11/site-packages/sklearn/utils/validation.py", line 1096, in check_array raise ValueError( ValueError: Found array with 1 feature(s) (shape=(200000, 1)) while a minimum of 2 is required by GraphicalLassoCV.

Spike sorting failed. You can inspect the runtime trace in /media/paul/Test/TY20210401_640_signalCheck_morning/processed/results_sc/0/spikeinterface_log.json.

Is there a way to shut off whitening? I do not see it as a sorter parameter.

I also tried to just run it as:

sorting_SC = si.run_sorter(sorter_name='spykingcircus2', recording=recording_cache, output_folder=base_folder+'processed/results_sc', verbose=True, **sorter_params)

I get a different type of error SpikeSortingError: Spike sorting error trace: Traceback (most recent call last): File "/home/paul/anaconda3/envs/sin/lib/python3.11/site-packages/spikeinterface/sorters/basesorter.py", line 261, in run_from_folder SorterClass._run_from_folder(sorter_output_folder, sorter_params, verbose) File "/home/paul/anaconda3/envs/sin/lib/python3.11/site-packages/spikeinterface/sorters/internal/spyking_circus2.py", line 229, in _run_from_folder labels, peak_labels = find_cluster_from_peaks( ^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/paul/anaconda3/envs/sin/lib/python3.11/site-packages/spikeinterface/sortingcomponents/clustering/main.py", line 44, in find_cluster_from_peaks outputs = method_class.main_function(recording, peaks, params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/paul/anaconda3/envs/sin/lib/python3.11/site-packages/spikeinterface/sortingcomponents/clustering/circus.py", line 245, in main_function probe=recording.get_probe(), ^^^^^^^^^^^^^^^^^^^^^ File "/home/paul/anaconda3/envs/sin/lib/python3.11/site-packages/spikeinterface/core/baserecordingsnippets.py", line 255, in get_probe assert len(probes) == 1, "there are several probe use .get_probes() or get_probegroup()" ^^^^^^^^^^^^^^^^ AssertionError: there are several probe use .get_probes() or get_probegroup()

Spike sorting failed. You can inspect the runtime trace in /media/paul/Test/TY20210401_640_signalCheck_morning/processed/results_sc/spikeinterface_log.json. [ ]:

Thank you for any help!

[yger]

In main now, regularization of the whitening is turned off by default in SC2 so the bug should be gone

[paul-aparicio]

Thanks! It was able to work to completion now. A warning message I am puzzled about that kept showing up was the following:

/media/paul/storage/pdev/spikeinterface/src/spikeinterface/core/job_tools.py:103: UserWarning: n_jobs is not set so parallel processing is disabled! To speed up computations, it is recommended to set n_jobs either globally (with the spikeinterface.set_global_job_kwargs() function) or locally (with the n_jobs argument). Use spikeinterface.set_global_job_kwargs? for more information about job_kwargs. warnings.warn(

It looks like the parameter has a default setting for n_jobs, so I don't understand the message. I tried setting n_jobs to different values in the parameters and also tried setting the global variable as suggested in the message, but I still got the same warning? Thanks again for your help.

[yger]

Yes, sorry, this will be fixed soon. Internally, there is one step in SC2 (after clustering), where the n_jobs arguments is not properly propagated and set to 1. I'll fix that, but this is not drastically harming the speed since this step is not the main bottleneck

[paul-aparicio]

Ok, Thanks!