Found array with 0 sample(s) (shape=(0, 80)) while a minimum of 1 is required by TruncatedSVD.

[AbhiSwamiUConn]

I had originally thought this issue had fixed itself a couple of weeks ago but it has occurred again. When using tridesclous2 to sort data, I get the following error below. When using spykingcircus2 however, my code runs without issues. Is this an problem on my end? I am using version 0.100.7. I noticed that in #2895, someone got a similar error when using kilosort4.

SpikeSortingError                         Traceback (most recent call last)
Cell In[6], line 2
      1 #Sort
----> 2 sorting = ss.run_sorter("tridesclous2", recording_f, output_folder= output_folder1, remove_existing_folder= True)
      3 Sanalyzer = si.waveform_extractor.extract_waveforms(recording=recording_f, sorting=sorting, folder = output_folder2, overwrite= True)

File ~\anaconda3\Lib\site-packages\spikeinterface\sorters\runsorter.py:175, in run_sorter(sorter_name, recording, output_folder, remove_existing_folder, delete_output_folder, verbose, raise_error, docker_image, singularity_image, delete_container_files, with_output, **sorter_params)
    168             container_image = singularity_image
    169     return run_sorter_container(
    170         container_image=container_image,
    171         mode=mode,
    172         **common_kwargs,
    173     )
--> 175 return run_sorter_local(**common_kwargs)

File ~\anaconda3\Lib\site-packages\spikeinterface\sorters\runsorter.py:225, in run_sorter_local(sorter_name, recording, output_folder, remove_existing_folder, delete_output_folder, verbose, raise_error, with_output, **sorter_params)
    223 SorterClass.set_params_to_folder(recording, output_folder, sorter_params, verbose)
    224 SorterClass.setup_recording(recording, output_folder, verbose=verbose)
--> 225 SorterClass.run_from_folder(output_folder, raise_error, verbose)
    226 if with_output:
    227     sorting = SorterClass.get_result_from_folder(output_folder, register_recording=True, sorting_info=True)

File ~\anaconda3\Lib\site-packages\spikeinterface\sorters\basesorter.py:293, in BaseSorter.run_from_folder(cls, output_folder, raise_error, verbose)
    290         print(f"{sorter_name} run time {run_time:0.2f}s")
    292 if has_error and raise_error:
--> 293     raise SpikeSortingError(
    294         f"Spike sorting error trace:\n{log['error_trace']}\n"
    295         f"Spike sorting failed. You can inspect the runtime trace in {output_folder}/spikeinterface_log.json."
    296     )
    298 return run_time

SpikeSortingError: Spike sorting error trace:
concurrent.futures.process._RemoteTraceback: 
"""
Traceback (most recent call last):
  File "C:\Users\User\anaconda3\Lib\concurrent\futures\process.py", line 261, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\User\anaconda3\Lib\concurrent\futures\process.py", line 210, in _process_chunk
    return [fn(*args) for args in chunk]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\User\anaconda3\Lib\concurrent\futures\process.py", line 210, in <listcomp>
    return [fn(*args) for args in chunk]
            ^^^^^^^^^
  File "C:\Users\User\anaconda3\Lib\site-packages\spikeinterface\core\job_tools.py", line 439, in function_wrapper
    return _func(segment_index, start_frame, end_frame, _worker_ctx)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\User\anaconda3\Lib\site-packages\spikeinterface\core\node_pipeline.py", line 522, in _compute_peak_pipeline_chunk
    node_output = node.compute(traces_chunk, *node_input_args)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\User\anaconda3\Lib\site-packages\spikeinterface\sortingcomponents\waveforms\temporal_pca.py", line 232, in compute
    projected_temporal_waveforms = self.pca_model.transform(temporal_waveforms)
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\User\anaconda3\Lib\site-packages\sklearn\utils\_set_output.py", line 140, in wrapped
    data_to_wrap = f(self, X, *args, **kwargs)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\User\anaconda3\Lib\site-packages\sklearn\decomposition\_truncated_svd.py", line 287, in transform
    X = self._validate_data(X, accept_sparse=["csr", "csc"], reset=False)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\User\anaconda3\Lib\site-packages\sklearn\base.py", line 565, in _validate_data
    X = check_array(X, input_name="X", **check_params)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\User\anaconda3\Lib\site-packages\sklearn\utils\validation.py", line 931, in check_array
    raise ValueError(
ValueError: Found array with 0 sample(s) (shape=(0, 80)) while a minimum of 1 is required by TruncatedSVD.
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\User\anaconda3\Lib\site-packages\spikeinterface\sorters\basesorter.py", line 258, in run_from_folder
    SorterClass._run_from_folder(sorter_output_folder, sorter_params, verbose)
  File "C:\Users\User\anaconda3\Lib\site-packages\spikeinterface\sorters\internal\tridesclous2.py", line 199, in _run_from_folder
    output = run_node_pipeline(
             ^^^^^^^^^^^^^^^^^^
  File "C:\Users\User\anaconda3\Lib\site-packages\spikeinterface\core\node_pipeline.py", line 471, in run_node_pipeline
    processor.run()
  File "C:\Users\User\anaconda3\Lib\site-packages\spikeinterface\core\job_tools.py", line 401, in run
    for res in results:
  File "C:\Users\User\anaconda3\Lib\concurrent\futures\process.py", line 620, in _chain_from_iterable_of_lists
    for element in iterable:
  File "C:\Users\User\anaconda3\Lib\concurrent\futures\_base.py", line 619, in result_iterator
    yield _result_or_cancel(fs.pop())
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\User\anaconda3\Lib\concurrent\futures\_base.py", line 317, in _result_or_cancel
    return fut.result(timeout)
           ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\User\anaconda3\Lib\concurrent\futures\_base.py", line 456, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\User\anaconda3\Lib\concurrent\futures\_base.py", line 401, in __get_result
    raise self._exception
ValueError: Found array with 0 sample(s) (shape=(0, 80)) while a minimum of 1 is required by TruncatedSVD.

Spike sorting failed. You can inspect the runtime trace in C:\Users\User\Desktop\Abhinav\files1/spikeinterface_log.json.

[zm711]

@yger @samuelgarcia seems like this is in sortingcomponents?

[samuelgarcia]

I think we fixed the problem one or two month ago but I do not remember which PR. I guess the main is OK.

[zm711]

@AbhiSwamiUConn did you test this on main? Is it working for you now?

[AbhiSwamiUConn]

Yes, it works and the issue is fixed.