Open sarahthon opened 3 months ago
sarahthon
commented
3 months ago I ran the spikesorting on autothreshold for bad channels at 1000, just to include all channels, and then it works. But of course we want to remove these. So that's where the problem lies.
alejoe91
commented
3 months ago Thanks @sarahthon
I think that the problem is that some clusters have too few spikes, so PCA is not fitted properly. Is it ok in general if I remove all clusters with less than 5 spikes? That should fix it :)
Let me know and I'll push a fix!
sarahthon
commented
3 months ago Yes that sounds good.
alejoe91
commented
3 months ago @sarahthon this commit https://github.com/CINPLA/expipe-plugin-cinpla/pull/64/commits/3497a51f7f901efbc3c5dfdab6dee08b20411ba6 should fix it.
@lepmik can you deploy on EduCloud?
Hi, I am suddenly recieving the following error when spikesorting two recordings made one the same day, however the spikesorting worked fine with the four recrodings taken just prior with the same settings. Can the problem be related to the kernel?
Processing 011-250321-4
Cleaning up existing NWB file
Preprocessing recording: Num channels: 32 Duration: 1210.81 s Detected bad channels: ['CH33' 'CH36' 'CH39' 'CH57' 'CH58' 'CH60' 'CH61'] Active channels: 25 Saving preprocessed recording
Spike sorting with mountainsort4 using installed sorter Found 34 units!
Postprocessing Extracting waveforms Computing QC metrics Exporting to phy ERROR: unable to process 011-250321-4 concurrent.futures.process._RemoteTraceback: """ Traceback (most recent call last): File "/projects/ec109/conda-envs/cinpla/lib/python3.11/concurrent/futures/process.py", line 261, in _process_worker r = call_item.fn(*call_item.args, call_item.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/projects/ec109/conda-envs/cinpla/lib/python3.11/concurrent/futures/process.py", line 210, in _process_chunk return [fn(args) for args in chunk] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/projects/ec109/conda-envs/cinpla/lib/python3.11/concurrent/futures/process.py", line 210, in
return [fn( args) for args in chunk]
^^^^^^^^^
File "/projects/ec109/conda-envs/cinpla/lib/python3.11/site-packages/spikeinterface/core/job_tools.py", line 439, in function_wrapper
return _func(segment_index, start_frame, end_frame, _worker_ctx)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/projects/ec109/conda-envs/cinpla/lib/python3.11/site-packages/spikeinterface/postprocessing/principal_component.py", line 661, in _all_pc_extractor_chunk
all_pcs[i, :, c] = pca_model[chan_ind].transform(w)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/projects/ec109/conda-envs/cinpla/lib/python3.11/site-packages/sklearn/utils/_set_output.py", line 295, in wrapped
data_to_wrap = f(self, X, *args, *kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/projects/ec109/conda-envs/cinpla/lib/python3.11/site-packages/sklearn/decomposition/_incremental_pca.py", line 409, in transform
return super().transform(X)
^^^^^^^^^^^^^^^^^^^^
File "/projects/ec109/conda-envs/cinpla/lib/python3.11/site-packages/sklearn/utils/_set_output.py", line 295, in wrapped
data_to_wrap = f(self, X, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/projects/ec109/conda-envs/cinpla/lib/python3.11/site-packages/sklearn/decomposition/_base.py", line 143, in transform
check_is_fitted(self)
File "/projects/ec109/conda-envs/cinpla/lib/python3.11/site-packages/sklearn/utils/validation.py", line 1622, in check_is_fitted
raise NotFittedError(msg % {"name": type(estimator).name})
sklearn.exceptions.NotFittedError: This IncrementalPCA instance is not fitted yet. Call 'fit' with appropriate arguments before using this estimator.
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/fp/projects01/ec109/software/expipe-plugin-cinpla/src/expipe_plugin_cinpla/widgets/process.py", line 296, in on_run process.process_ecephys( File "/fp/projects01/ec109/software/expipe-plugin-cinpla/src/expipe_plugin_cinpla/scripts/process.py", line 276, in process_ecephys sexp.export_to_phy( File "/projects/ec109/conda-envs/cinpla/lib/python3.11/site-packages/spikeinterface/exporters/to_phy.py", line 245, in export_to_phy pc.run_for_all_spikes(output_folder / "pc_features.npy", **job_kwargs) File "/projects/ec109/conda-envs/cinpla/lib/python3.11/site-packages/spikeinterface/postprocessing/principal_component.py", line 373, in run_for_all_spikes processor.run() File "/projects/ec109/conda-envs/cinpla/lib/python3.11/site-packages/spikeinterface/core/job_tools.py", line 401, in run for res in results: File "/projects/ec109/conda-envs/cinpla/lib/python3.11/concurrent/futures/process.py", line 620, in _chain_from_iterable_of_lists for element in iterable: File "/projects/ec109/conda-envs/cinpla/lib/python3.11/concurrent/futures/_base.py", line 619, in result_iterator yield _result_or_cancel(fs.pop()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/projects/ec109/conda-envs/cinpla/lib/python3.11/concurrent/futures/_base.py", line 317, in _result_or_cancel return fut.result(timeout) ^^^^^^^^^^^^^^^^^^^ File "/projects/ec109/conda-envs/cinpla/lib/python3.11/concurrent/futures/_base.py", line 456, in result return self.get_result() ^^^^^^^^^^^^^^^^^^^ File "/projects/ec109/conda-envs/cinpla/lib/python3.11/concurrent/futures/_base.py", line 401, in get_result raise self._exception sklearn.exceptions.NotFittedError: This IncrementalPCA instance is not fitted yet. Call 'fit' with appropriate arguments before using this estimator.