search

SpikeInterface / spikeinterface

A Python-based module for creating flexible and robust spike sorting pipelines.
https://spikeinterface.readthedocs.io
MIT License
533 stars 187 forks source link

Issues Extracting Waveforms for Subset of Units #2401

Closed litzj01 closed 10 months ago

litzj01 commented 10 months ago

I am getting an error when trying to extract the waveforms only for a subset of units. I have recordings with huge numbers of units and spikes, that make it impossible to extract the waveforms for all units without having extremely large file sizes (even with sparsity). I saw someone had raised a similar issue (#330), and the suggested solution was to create a sub-sorter object first which is what I am trying to do, but it is not working.

Here is the code I am running:

#Get recording object 
recording = se.read_biocam('D:/Temporary_Storage_for_Cdrive/060623_Primate/06.06.23_Primate_Reye_OD_ST_Stimuli_01.brw') 

#Get sorting object 
sorting = ss.read_sorter_folder(output_folder='D:/Temporary_Storage_for_Cdrive/060623_Primate/herdingspikes_output', register_recording=True, sorting_info=True) 

#Load unit ids 
unit_ids=sorting.unit_ids 

#Load channel ids 
channel_ids=recording.channel_ids 

#Create array of subset of units
OnCells = [1,4,6,7,10,13,19,20,24,27,30,36,38,44,46,47,51,54,70,71,75,80,94,104,107,110,118,137,139,153,175,182,184,189,213,217,218,224,226,233,238,248,284,298,301,307,327,331,378,404,487]
OffCells = [0,42,56,59,66,89,108,114,116,133,149,155,156,158,160,174,176,178,190,192,193,203,209,228,253,268,283,296,316,338,343,361,369,403,412,414,423,449,451,484,523,569,586,823]
OnOffCells =[2,3,5,8,11,17,18,22,23,25,26,29,35,37,39,40,48,50,52,55,62,67,69,74,76,77,79,83,85,98,99,105,111,112,115,117,132,135,140,141,142,143,145,150,164,167,168,169,171,179,183,185,186,188,191,195,198,200,205,208,216,220,223,237,247,251,254,255,256,258,278,280,282,302,333,337,349,377,391,405,455,497,639,4297,6582,7343]

#Create new sorting objects with unit subsets
sortingON=sorting.select_units(unit_ids=OnCells)
sortingON.save(folder="D:/Temporary_Storage_for_Cdrive/060623_Primate/" + 'OnCells')

sortingOFF=sorting.select_units(unit_ids=OffCells)
sortingOFF.save(folder="D:/Temporary_Storage_for_Cdrive/060623_Primate/" + 'OffCells')

sortingONOFF=sorting.select_units(unit_ids=OnOffCells)
sortingONOFF.save(folder="D:/Temporary_Storage_for_Cdrive/060623_Primate/" + 'OnOffCells', )

#Extract the waveforms for the On cells
we_sparseON = si.extract_waveforms(recording, sortingON, "D:/Temporary_Storage_for_Cdrive/060623_Primate/OnCells/" + 'waveforms', overwrite=True, allow_unfiltered=True, sparse=True, method="radius", radius_um=70, **global_job_kwargs) 
print("Extracting waveforms:", we_sparseON)
# %%
amplitudes = spost.compute_spike_amplitudes(we_sparseON) 
unit_locations = spost.compute_unit_locations(we_sparseON) 
spike_locations = spost.compute_spike_locations(we_sparseON) 
correlograms, bins = spost.compute_correlograms(we_sparseON) 
similarity = spost.compute_template_similarity(we_sparseON)

Then I get the following error:


BrokenProcessPool                         Traceback (most recent call last)
Cell In[13], [line 1](vscode-notebook-cell:?execution_count=13&line=1)
----> [1](vscode-notebook-cell:?execution_count=13&line=1) we_sparseON = si.extract_waveforms(recording, sortingON, "[D:/Temporary_Storage_for_Cdrive/060623_Primate/OnCells/](file:///D:/Temporary_Storage_for_Cdrive/060623_Primate/OnCells/)" + 'waveforms', overwrite=True, allow_unfiltered=True, sparse=True, method="radius", radius_um=70, **global_job_kwargs, unit_ids=OnCells) 
      [2](vscode-notebook-cell:?execution_count=13&line=2) print("Extracting waveforms:", we_sparseON)
      [3](vscode-notebook-cell:?execution_count=13&line=3) # %%

File [c:\Users\litzj\miniconda\envs\spikeinterface\lib\site-packages\spikeinterface\core\waveform_extractor.py:1593](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1593), in extract_waveforms(recording, sorting, folder, mode, precompute_template, ms_before, ms_after, max_spikes_per_unit, overwrite, return_scaled, dtype, sparse, sparsity, num_spikes_for_sparsity, allow_unfiltered, use_relative_path, seed, load_if_exists, **kwargs)
   [1591](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1591)         assert all(ch in recording.channel_ids for ch in channels), "Invalid channel ids in sparsity"
   [1592](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1592) elif sparse:
-> [1593](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1593)     sparsity = precompute_sparsity(
   [1594](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1594)         recording,
   [1595](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1595)         sorting,
   [1596](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1596)         ms_before=ms_before,
   [1597](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1597)         ms_after=ms_after,
   [1598](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1598)         num_spikes_for_sparsity=num_spikes_for_sparsity,
   [1599](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1599)         allow_unfiltered=allow_unfiltered,
   [1600](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1600)         **estimate_kwargs,
   [1601](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1601)         **job_kwargs,
   [1602](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1602)     )
   [1603](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1603) else:
   [1604](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1604)     sparsity = None

File [c:\Users\litzj\miniconda\envs\spikeinterface\lib\site-packages\spikeinterface\core\waveform_extractor.py:1718](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1718), in precompute_sparsity(recording, sorting, num_spikes_for_sparsity, unit_batch_size, ms_before, ms_after, allow_unfiltered, **kwargs)
   [1716](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1716) local_ids = unit_ids[sl]
   [1717](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1717) local_sorting = sorting.select_units(local_ids)
-> [1718](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1718) local_we = extract_waveforms(
   [1719](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1719)     recording,
   [1720](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1720)     local_sorting,
   [1721](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1721)     folder=None,
   [1722](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1722)     mode="memory",
   [1723](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1723)     precompute_template=("average",),
   [1724](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1724)     ms_before=ms_before,
   [1725](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1725)     ms_after=ms_after,
   [1726](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1726)     max_spikes_per_unit=num_spikes_for_sparsity,
   [1727](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1727)     return_scaled=False,
   [1728](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1728)     allow_unfiltered=allow_unfiltered,
   [1729](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1729)     sparse=False,
   [1730](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1730)     **job_kwargs,
   [1731](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1731) )
   [1732](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1732) local_sparsity = compute_sparsity(local_we, **sparse_kwargs)
   [1733](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1733) mask[sl, :] = local_sparsity.mask

File [c:\Users\litzj\miniconda\envs\spikeinterface\lib\site-packages\spikeinterface\core\waveform_extractor.py:1622](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1622), in extract_waveforms(recording, sorting, folder, mode, precompute_template, ms_before, ms_after, max_spikes_per_unit, overwrite, return_scaled, dtype, sparse, sparsity, num_spikes_for_sparsity, allow_unfiltered, use_relative_path, seed, load_if_exists, **kwargs)
   [1606](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1606) we = WaveformExtractor.create(
   [1607](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1607)     recording,
   [1608](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1608)     sorting,
   (...)
   [1613](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1613)     sparsity=sparsity,
   [1614](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1614) )
   [1615](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1615) we.set_params(
   [1616](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1616)     ms_before=ms_before,
   [1617](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1617)     ms_after=ms_after,
   (...)
   [1620](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1620)     return_scaled=return_scaled,
   [1621](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1621) )
-> [1622](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1622) we.run_extract_waveforms(seed=seed, **job_kwargs)
   [1624](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1624) if precompute_template is not None:
   [1625](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1625)     we.precompute_templates(modes=precompute_template)

File [c:\Users\litzj\miniconda\envs\spikeinterface\lib\site-packages\spikeinterface\core\waveform_extractor.py:1391](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1391), in WaveformExtractor.run_extract_waveforms(self, seed, **job_kwargs)
   [1388](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1388) else:
   [1389](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1389)     sparsity_mask = self.sparsity.mask
-> [1391](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1391) wfs_arrays = extract_waveforms_to_buffers(
   [1392](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1392)     self.recording,
   [1393](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1393)     spikes,
   [1394](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1394)     unit_ids,
   [1395](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1395)     nbefore,
   [1396](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1396)     nafter,
   [1397](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1397)     mode=mode,
   [1398](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1398)     return_scaled=return_scaled,
   [1399](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1399)     folder=wf_folder,
   [1400](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1400)     dtype=p["dtype"],
   [1401](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1401)     sparsity_mask=sparsity_mask,
   [1402](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1402)     copy=copy,
   [1403](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1403)     **job_kwargs,
   [1404](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1404) )
   [1405](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1405) if self.folder is None:
   [1406](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_extractor.py:1406)     self._memory_objects["wfs_arrays"] = wfs_arrays

File [c:\Users\litzj\miniconda\envs\spikeinterface\lib\site-packages\spikeinterface\core\waveform_tools.py:96](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_tools.py:96), in extract_waveforms_to_buffers(recording, spikes, unit_ids, nbefore, nafter, mode, return_scaled, folder, dtype, sparsity_mask, copy, **job_kwargs)
     [90](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_tools.py:90) dtype = np.dtype(dtype)
     [92](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_tools.py:92) waveforms_by_units, arrays_info = allocate_waveforms_buffers(
     [93](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_tools.py:93)     recording, spikes, unit_ids, nbefore, nafter, mode=mode, folder=folder, dtype=dtype, sparsity_mask=sparsity_mask
     [94](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_tools.py:94) )
---> [96](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_tools.py:96) distribute_waveforms_to_buffers(
     [97](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_tools.py:97)     recording,
     [98](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_tools.py:98)     spikes,
     [99](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_tools.py:99)     unit_ids,
    [100](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_tools.py:100)     arrays_info,
    [101](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_tools.py:101)     nbefore,
    [102](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_tools.py:102)     nafter,
    [103](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_tools.py:103)     return_scaled,
    [104](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_tools.py:104)     mode=mode,
    [105](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_tools.py:105)     sparsity_mask=sparsity_mask,
    [106](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_tools.py:106)     **job_kwargs,
    [107](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_tools.py:107) )
    [109](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_tools.py:109) if mode == "memmap":
    [110](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_tools.py:110)     return waveforms_by_units

File [c:\Users\litzj\miniconda\envs\spikeinterface\lib\site-packages\spikeinterface\core\waveform_tools.py:279](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_tools.py:279), in distribute_waveforms_to_buffers(recording, spikes, unit_ids, arrays_info, nbefore, nafter, return_scaled, mode, sparsity_mask, job_name, **job_kwargs)
    [277](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_tools.py:277)     job_name = f"extract waveforms {mode} multi buffer"
    [278](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_tools.py:278) processor = ChunkRecordingExecutor(recording, func, init_func, init_args, job_name=job_name, **job_kwargs)
--> [279](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/waveform_tools.py:279) processor.run()

File [c:\Users\litzj\miniconda\envs\spikeinterface\lib\site-packages\spikeinterface\core\job_tools.py:391](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/job_tools.py:391), in ChunkRecordingExecutor.run(self)
    [384](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/job_tools.py:384) # parallel
    [385](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/job_tools.py:385) with ProcessPoolExecutor(
    [386](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/job_tools.py:386)     max_workers=n_jobs,
    [387](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/job_tools.py:387)     initializer=worker_initializer,
    [388](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/job_tools.py:388)     mp_context=mp.get_context(self.mp_context),
    [389](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/job_tools.py:389)     initargs=(self.func, self.init_func, self.init_args, self.max_threads_per_process),
    [390](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/job_tools.py:390) ) as executor:
--> [391](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/job_tools.py:391)     results = executor.map(function_wrapper, all_chunks)
    [393](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/job_tools.py:393)     if self.progress_bar:
    [394](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/site-packages/spikeinterface/core/job_tools.py:394)         results = tqdm(results, desc=self.job_name, total=len(all_chunks))

File [c:\Users\litzj\miniconda\envs\spikeinterface\lib\concurrent\futures\process.py:766](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/concurrent/futures/process.py:766), in ProcessPoolExecutor.map(self, fn, timeout, chunksize, *iterables)
    [763](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/concurrent/futures/process.py:763) if chunksize < 1:
    [764](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/concurrent/futures/process.py:764)     raise ValueError("chunksize must be >= 1.")
--> [766](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/concurrent/futures/process.py:766) results = super().map(partial(_process_chunk, fn),
    [767](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/concurrent/futures/process.py:767)                       _get_chunks(*iterables, chunksize=chunksize),
    [768](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/concurrent/futures/process.py:768)                       timeout=timeout)
    [769](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/concurrent/futures/process.py:769) return _chain_from_iterable_of_lists(results)

File [c:\Users\litzj\miniconda\envs\spikeinterface\lib\concurrent\futures\_base.py:610](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/concurrent/futures/_base.py:610), in Executor.map(self, fn, timeout, chunksize, *iterables)
    [607](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/concurrent/futures/_base.py:607) if timeout is not None:
    [608](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/concurrent/futures/_base.py:608)     end_time = timeout + time.monotonic()
--> [610](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/concurrent/futures/_base.py:610) fs = [self.submit(fn, *args) for args in zip(*iterables)]
    [612](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/concurrent/futures/_base.py:612) # Yield must be hidden in closure so that the futures are submitted
    [613](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/concurrent/futures/_base.py:613) # before the first iterator value is required.
    [614](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/concurrent/futures/_base.py:614) def result_iterator():

File [c:\Users\litzj\miniconda\envs\spikeinterface\lib\concurrent\futures\_base.py:610](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/concurrent/futures/_base.py:610), in <listcomp>(.0)
    [607](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/concurrent/futures/_base.py:607) if timeout is not None:
    [608](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/concurrent/futures/_base.py:608)     end_time = timeout + time.monotonic()
--> [610](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/concurrent/futures/_base.py:610) fs = [self.submit(fn, *args) for args in zip(*iterables)]
    [612](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/concurrent/futures/_base.py:612) # Yield must be hidden in closure so that the futures are submitted
    [613](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/concurrent/futures/_base.py:613) # before the first iterator value is required.
    [614](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/concurrent/futures/_base.py:614) def result_iterator():

File [c:\Users\litzj\miniconda\envs\spikeinterface\lib\concurrent\futures\process.py:720](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/concurrent/futures/process.py:720), in ProcessPoolExecutor.submit(self, fn, *args, **kwargs)
    [718](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/concurrent/futures/process.py:718) with self._shutdown_lock:
    [719](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/concurrent/futures/process.py:719)     if self._broken:
--> [720](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/concurrent/futures/process.py:720)         raise BrokenProcessPool(self._broken)
    [721](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/concurrent/futures/process.py:721)     if self._shutdown_thread:
    [722](file:///C:/Users/litzj/miniconda/envs/spikeinterface/lib/concurrent/futures/process.py:722)         raise RuntimeError('cannot schedule new futures after shutdown')

BrokenProcessPool: A child process terminated abruptly, the process pool is not usable anymore
zm711 commented 10 months ago

@litzj01

What are the job_kwargs you are using? What is the version of spikeinterface? What is the version number of threadpoolctl? (ie you can just do a conda list in your spikeinterface environment and post that all here).

And based on your initial statement you were able to get this to work on your full data, but it was too big (the result files took up too much space I mean) or even if you try your full dataset it fails?

litzj01 commented 10 months ago

Hello, thanks for your reply! The spike interface version is 0.99.0.dev0. The job_kwargs are just

 global_job_kwargs = dict(n_jobs=12) 
si.set_global_job_kwargs(**global_job_kwargs)

I have successfully extracted the waveforms for all units on other slightly smaller datasets. However, this raw data file is 92 GB and there are 12,374 detected units with over 14 million spikes total. I did try extracting the waveforms for all units in this dataset, but it failed because the code was trying to use too many threads which exceeded the limit my ssh server allows, so it likely wasn't related to this issue. Now because of that limitation, I am trying to extract the subset of waveforms on my local computer after running the sort in a ssh client and downloading the sorter output.

Here are all the packages in my environment:

# Name                    Version                   Build  Channel
aiohttp                   3.8.6                    pypi_0    pypi
aiosignal                 1.3.1                    pypi_0    pypi
altair                    5.1.2                    pypi_0    pypi
asciitree                 0.3.3                    pypi_0    pypi
asttokens                 2.4.0              pyhd8ed1ab_0    conda-forge
async-timeout             4.0.3                    pypi_0    pypi
attrs                     23.1.0                   pypi_0    pypi
backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
backports                 1.0                pyhd8ed1ab_3    conda-forge
backports.functools_lru_cache 1.6.5              pyhd8ed1ab_0    conda-forge
blas                      1.0                         mkl
bottleneck                1.3.5           py310h9128911_0
brotli                    1.0.9                h2bbff1b_7
brotli-bin                1.0.9                h2bbff1b_7
bzip2                     1.0.8                he774522_0
ca-certificates           2023.7.22            h56e8100_0    conda-forge
cbor2                     5.5.1                    pypi_0    pypi
certifi                   2023.7.22                pypi_0    pypi
cffi                      1.16.0                   pypi_0    pypi
charset-normalizer        3.3.0                    pypi_0    pypi
click                     8.1.7                    pypi_0    pypi
cloudpickle               3.0.0                    pypi_0    pypi
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
comm                      0.1.4              pyhd8ed1ab_0    conda-forge
contourpy                 1.0.5           py310h59b6b97_0
cryptography              41.0.5                   pypi_0    pypi
cuda-python               12.2.0                   pypi_0    pypi
cycler                    0.11.0             pyhd3eb1b0_0
cython                    3.0.0           py310h2bbff1b_0
dask                      2023.10.1                pypi_0    pypi
debugpy                   1.6.7           py310hd77b12b_0
decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
distinctipy               1.2.3                    pypi_0    pypi
distributed               2023.10.1                pypi_0    pypi
exceptiongroup            1.1.3              pyhd8ed1ab_0    conda-forge
executing                 1.2.0              pyhd8ed1ab_0    conda-forge
fasteners                 0.19                     pypi_0    pypi
figurl                    0.2.18                   pypi_0    pypi
figurl-jupyter            0.2.8                    pypi_0    pypi
fonttools                 4.25.0             pyhd3eb1b0_0
freetype                  2.12.1               ha860e81_0
frozenlist                1.4.0                    pypi_0    pypi
fsspec                    2023.10.0                pypi_0    pypi
giflib                    5.2.1                h8cc25b3_3
glib                      2.69.1               h5dc1a3c_2
h5py                      3.9.0           py310hfc34f40_0
hdf5                      1.12.1               h51c971a_3
hdmf                      3.11.0                   pypi_0    pypi
herdingspikes             0.3.102                  pypi_0    pypi
icc_rt                    2022.1.0             h6049295_2
icu                       58.2                 ha925a31_3
idna                      3.4                      pypi_0    pypi
importlib-metadata        6.8.0              pyha770c72_0    conda-forge
importlib_metadata        6.8.0                hd8ed1ab_0    conda-forge
intel-openmp              2023.1.0         h59b6b97_46319
ipykernel                 6.25.2             pyh60829e3_0    conda-forge
ipython                   8.16.1             pyh5737063_0    conda-forge
ipywidgets                8.1.1                    pypi_0    pypi
jedi                      0.19.1             pyhd8ed1ab_0    conda-forge
jinja2                    3.1.2                    pypi_0    pypi
joblib                    1.2.0           py310haa95532_0
jpeg                      9e                   h2bbff1b_1
jsonschema                4.19.2                   pypi_0    pypi
jsonschema-specifications 2023.7.1                 pypi_0    pypi
jupyter_client            8.4.0              pyhd8ed1ab_0    conda-forge
jupyter_core              5.4.0           py310h5588dad_0    conda-forge
jupyterlab-widgets        3.0.9                    pypi_0    pypi
kachery-cloud             0.4.3                    pypi_0    pypi
kiwisolver                1.4.4           py310hd77b12b_0
krb5                      1.20.1               h5b6d351_0
lerc                      3.0                  hd77b12b_0
libbrotlicommon           1.0.9                h2bbff1b_7
libbrotlidec              1.0.9                h2bbff1b_7
libbrotlienc              1.0.9                h2bbff1b_7
libclang                  14.0.6          default_hb5a9fac_1
libclang13                14.0.6          default_h8e68704_1
libdeflate                1.17                 h2bbff1b_1
libffi                    3.4.4                hd77b12b_0
libiconv                  1.16                 h2bbff1b_2
libpng                    1.6.39               h8cc25b3_0
libpq                     12.15                h906ac69_1
libsodium                 1.0.18               h8d14728_1    conda-forge
libtiff                   4.5.1                hd77b12b_0
libwebp                   1.3.2                hbc33d0d_0
libwebp-base              1.3.2                h2bbff1b_0
libxml2                   2.10.4               h0ad7f3c_1
libxslt                   1.1.37               h2bbff1b_1
llvmlite                  0.41.0                   pypi_0    pypi
locket                    1.0.0                    pypi_0    pypi
lz4-c                     1.9.4                h2bbff1b_0
markupsafe                2.1.3                    pypi_0    pypi
matplotlib                3.7.2           py310haa95532_0
matplotlib-base           3.7.2           py310h4ed8f06_0
matplotlib-inline         0.1.6              pyhd8ed1ab_0    conda-forge
meautility                1.5.1                    pypi_0    pypi
mkl                       2023.1.0         h6b88ed4_46357
mkl-service               2.4.0           py310h2bbff1b_1
mkl_fft                   1.3.8           py310h2bbff1b_0
mkl_random                1.2.4           py310h59b6b97_0
msgpack                   1.0.7                    pypi_0    pypi
multidict                 6.0.4                    pypi_0    pypi
munkres                   1.1.4                      py_0
neo                       0.12.0                   pypi_0    pypi
nest-asyncio              1.5.8              pyhd8ed1ab_0    conda-forge
networkx                  3.1             py310haa95532_0
numba                     0.58.0                   pypi_0    pypi
numcodecs                 0.12.0                   pypi_0    pypi
numexpr                   2.8.7           py310h2cd9be0_0
numpy                     1.25.2                   pypi_0    pypi
openjpeg                  2.4.0                h4fc8c34_0
openssl                   3.0.11               h2bbff1b_2
packaging                 23.1            py310haa95532_0
pandas                    2.0.3           py310h4ed8f06_0
parso                     0.8.3              pyhd8ed1ab_0    conda-forge
partd                     1.4.1                    pypi_0    pypi
pcre                      8.45                 hd77b12b_0
pickleshare               0.7.5                   py_1003    conda-forge
pillow                    10.0.1          py310h045eedc_0
pip                       23.2.1          py310haa95532_0
platformdirs              3.11.0             pyhd8ed1ab_0    conda-forge
ply                       3.11            py310haa95532_0
probeinterface            0.2.17                   pypi_0    pypi
prompt-toolkit            3.0.39             pyha770c72_0    conda-forge
prompt_toolkit            3.0.39               hd8ed1ab_0    conda-forge
psutil                    5.9.0           py310h2bbff1b_0
pubnub                    7.3.1                    pypi_0    pypi
pure_eval                 0.2.2              pyhd8ed1ab_0    conda-forge
pycparser                 2.21                     pypi_0    pypi
pycryptodomex             3.19.0                   pypi_0    pypi
pygments                  2.16.1             pyhd8ed1ab_0    conda-forge
pynwb                     2.5.0                    pypi_0    pypi
pyparsing                 3.0.9           py310haa95532_0
pyqt                      5.15.7          py310hd77b12b_0
pyqt5-sip                 12.11.0         py310hd77b12b_0
pyqtgraph                 0.13.3                   pypi_0    pypi
pyside6                   6.5.3                    pypi_0    pypi
pyside6-addons            6.5.3                    pypi_0    pypi
pyside6-essentials        6.5.3                    pypi_0    pypi
python                    3.10.13              he1021f5_0
python-dateutil           2.8.2              pyhd3eb1b0_0
python-tzdata             2023.3             pyhd3eb1b0_0
python_abi                3.10                    2_cp310    conda-forge
pytz                      2023.3.post1    py310haa95532_0
pywin32                   305             py310h2bbff1b_0
pyyaml                    6.0.1                    pypi_0    pypi
pyzmq                     23.2.1          py310h73ada01_0    conda-forge
qt-main                   5.15.2               h879a1e9_9
qt-webengine              5.15.9               h5bd16bc_7
qtwebkit                  5.212                h2bbfb41_5
quantities                0.14.1                   pypi_0    pypi
referencing               0.30.2                   pypi_0    pypi
requests                  2.31.0                   pypi_0    pypi
rpds-py                   0.12.0                   pypi_0    pypi
ruamel-yaml               0.18.5                   pypi_0    pypi
ruamel-yaml-clib          0.2.8                    pypi_0    pypi
scikit-learn              1.3.0           py310h4ed8f06_0
scipy                     1.11.3          py310h309d312_0
setuptools                68.0.0          py310haa95532_0
shiboken6                 6.5.3                    pypi_0    pypi
simplejson                3.19.2                   pypi_0    pypi
sip                       6.6.2           py310hd77b12b_0
six                       1.16.0             pyhd3eb1b0_1
sortedcontainers          2.4.0                    pypi_0    pypi
sortingview               0.12.0                   pypi_0    pypi
spikecomparison           0.3.3                    pypi_0    pypi
spikeextractors           0.9.11                   pypi_0    pypi
spikefeatures             0.1.2                    pypi_0    pypi
spikeinterface            0.99.0.dev0              pypi_0    pypi
spikeinterface-gui        0.7.0                    pypi_0    pypi
spikemetrics              0.2.4                    pypi_0    pypi
spikesorters              0.4.5                    pypi_0    pypi
spiketoolkit              0.7.7                    pypi_0    pypi
spikewidgets              0.5.4                    pypi_0    pypi
sqlite                    3.41.2               h2bbff1b_0
stack_data                0.6.2              pyhd8ed1ab_0    conda-forge
tbb                       2021.8.0             h59b6b97_0
tblib                     3.0.0                    pypi_0    pypi
threadpoolctl             2.2.0              pyh0d69192_0
tk                        8.6.12               h2bbff1b_0
toml                      0.10.2             pyhd3eb1b0_0
toolz                     0.12.0                   pypi_0    pypi
tornado                   6.3.3           py310h2bbff1b_0
tqdm                      4.66.1                   pypi_0    pypi
traitlets                 5.11.2             pyhd8ed1ab_0    conda-forge
typing-extensions         4.8.0                hd8ed1ab_0    conda-forge
typing_extensions         4.8.0              pyha770c72_0    conda-forge
tzdata                    2023c                h04d1e81_0
urllib3                   2.0.6                    pypi_0    pypi
vc                        14.2                 h21ff451_1
vs2015_runtime            14.27.29016          h5e58377_2
wcwidth                   0.2.8              pyhd8ed1ab_0    conda-forge
wheel                     0.41.2          py310haa95532_0
widgetsnbextension        4.0.9                    pypi_0    pypi
xarray                    2023.9.0                 pypi_0    pypi
xz                        5.4.2                h8cc25b3_0
yarl                      1.9.2                    pypi_0    pypi
zarr                      2.16.1                   pypi_0    pypi
zeromq                    4.3.4                h0e60522_1    conda-forge
zict                      3.0.0                    pypi_0    pypi
zipp                      3.17.0             pyhd8ed1ab_0    conda-forge
zlib                      1.2.13               h8cc25b3_0
zstd                      1.5.5                hd43e919_0
zm711 commented 10 months ago

@litzj01 could you try to update threadpoolctl? You can try:

conda update threadpoolctl

We want the version for that to be > 3.x.

Then retry your subset of data.

litzj01 commented 10 months ago

I just ran conda update threadpoolctl, but it seems to not have worked properly. I now get an error trying to initialize spike interface (import spikeinterface.full as si) with the errorImportError: Numba needs NumPy 1.25 or less.

Here is the new conda list:

aiohttp                   3.8.6                    pypi_0    pypi
aiosignal                 1.3.1                    pypi_0    pypi
altair                    5.1.2                    pypi_0    pypi
asciitree                 0.3.3                    pypi_0    pypi
asttokens                 2.0.5              pyhd3eb1b0_0
async-timeout             4.0.3                    pypi_0    pypi
attrs                     23.1.0                   pypi_0    pypi
backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
backports                 1.0                pyhd8ed1ab_3    conda-forge
backports.functools_lru_cache 1.6.5              pyhd8ed1ab_0    conda-forge
blas                      1.0                         mkl
bottleneck                1.3.5           py310h9128911_0
brotli                    1.0.9                h2bbff1b_7
brotli-bin                1.0.9                h2bbff1b_7
bzip2                     1.0.8                he774522_0
ca-certificates           2023.12.12           haa95532_0
cbor2                     5.5.1                    pypi_0    pypi
certifi                   2023.7.22                pypi_0    pypi
cffi                      1.16.0                   pypi_0    pypi
charset-normalizer        3.3.0                    pypi_0    pypi
click                     8.1.7                    pypi_0    pypi
cloudpickle               3.0.0                    pypi_0    pypi
colorama                  0.4.6           py310haa95532_0
comm                      0.1.2           py310haa95532_0
contourpy                 1.0.5           py310h59b6b97_0
cryptography              41.0.5                   pypi_0    pypi
cuda-python               12.2.0                   pypi_0    pypi
cycler                    0.11.0             pyhd3eb1b0_0
cython                    3.0.0           py310h2bbff1b_0
dask                      2023.10.1                pypi_0    pypi
debugpy                   1.6.7           py310hd77b12b_0
decorator                 5.1.1              pyhd3eb1b0_0
distinctipy               1.2.3                    pypi_0    pypi
distributed               2023.10.1                pypi_0    pypi
exceptiongroup            1.0.4           py310haa95532_0
executing                 0.8.3              pyhd3eb1b0_0
fasteners                 0.19                     pypi_0    pypi
figurl                    0.2.18                   pypi_0    pypi
figurl-jupyter            0.2.8                    pypi_0    pypi
fonttools                 4.25.0             pyhd3eb1b0_0
freetype                  2.12.1               ha860e81_0
frozenlist                1.4.0                    pypi_0    pypi
fsspec                    2023.10.0                pypi_0    pypi
giflib                    5.2.1                h8cc25b3_3
glib                      2.69.1               h5dc1a3c_2
h5py                      3.9.0           py310hfc34f40_0
hdf5                      1.12.1               h51c971a_3
hdmf                      3.11.0                   pypi_0    pypi
herdingspikes             0.3.102                  pypi_0    pypi
icc_rt                    2022.1.0             h6049295_2
icu                       73.1                 h6c2663c_0
idna                      3.4                      pypi_0    pypi
importlib-metadata        6.8.0              pyha770c72_0    conda-forge
importlib_metadata        6.8.0                hd8ed1ab_0    conda-forge
intel-openmp              2023.1.0         h59b6b97_46320
ipykernel                 6.25.2             pyh60829e3_0    conda-forge
ipython                   8.20.0          py310haa95532_0
ipywidgets                8.1.1                    pypi_0    pypi
jedi                      0.18.1          py310haa95532_1
jinja2                    3.1.2                    pypi_0    pypi
joblib                    1.2.0           py310haa95532_0
jpeg                      9e                   h2bbff1b_1
jsonschema                4.19.2                   pypi_0    pypi
jsonschema-specifications 2023.7.1                 pypi_0    pypi
jupyter_client            8.6.0           py310haa95532_0
jupyter_core              5.5.0           py310haa95532_0
jupyterlab-widgets        3.0.9                    pypi_0    pypi
kachery-cloud             0.4.3                    pypi_0    pypi
kiwisolver                1.4.4           py310hd77b12b_0
krb5                      1.20.1               h5b6d351_0
lerc                      3.0                  hd77b12b_0
libbrotlicommon           1.0.9                h2bbff1b_7
libbrotlidec              1.0.9                h2bbff1b_7
libbrotlienc              1.0.9                h2bbff1b_7
libclang                  14.0.6          default_hb5a9fac_1
libclang13                14.0.6          default_h8e68704_1
libdeflate                1.17                 h2bbff1b_1
libffi                    3.4.4                hd77b12b_0
libiconv                  1.16                 h2bbff1b_2
libpng                    1.6.39               h8cc25b3_0
libpq                     12.15                h906ac69_1
libsodium                 1.0.18               h62dcd97_0
libtiff                   4.5.1                hd77b12b_0
libwebp                   1.3.2                hbc33d0d_0
libwebp-base              1.3.2                h2bbff1b_0
libxml2                   2.10.4               h0ad7f3c_1
libxslt                   1.1.37               h2bbff1b_1
llvmlite                  0.41.0                   pypi_0    pypi
locket                    1.0.0                    pypi_0    pypi
lz4-c                     1.9.4                h2bbff1b_0
markupsafe                2.1.3                    pypi_0    pypi
matplotlib                3.7.2           py310haa95532_0
matplotlib-base           3.7.2           py310h4ed8f06_0
matplotlib-inline         0.1.6           py310haa95532_0
meautility                1.5.1                    pypi_0    pypi
mkl                       2023.1.0         h6b88ed4_46358
mkl-service               2.4.0           py310h2bbff1b_1
mkl_fft                   1.3.8           py310h2bbff1b_0
mkl_random                1.2.4           py310h59b6b97_0
msgpack                   1.0.7                    pypi_0    pypi
multidict                 6.0.4                    pypi_0    pypi
munkres                   1.1.4                      py_0
neo                       0.12.0                   pypi_0    pypi
nest-asyncio              1.5.6           py310haa95532_0
networkx                  3.1             py310haa95532_0
numba                     0.58.0                   pypi_0    pypi
numcodecs                 0.12.0                   pypi_0    pypi
numexpr                   2.8.7           py310h2cd9be0_0
numpy                     1.25.2                   pypi_0    pypi
numpy-base                1.26.3          py310h65a83cf_0
openjpeg                  2.4.0                h4fc8c34_0
openssl                   3.0.12               h2bbff1b_0
packaging                 23.1            py310haa95532_0
pandas                    2.0.3           py310h4ed8f06_0
parso                     0.8.3              pyhd3eb1b0_0
partd                     1.4.1                    pypi_0    pypi
pcre                      8.45                 hd77b12b_0
pickleshare               0.7.5                   py_1003    conda-forge
pillow                    10.0.1          py310h045eedc_0
pip                       23.3.1          py310haa95532_0
platformdirs              3.10.0          py310haa95532_0
ply                       3.11            py310haa95532_0
probeinterface            0.2.17                   pypi_0    pypi
prompt-toolkit            3.0.43          py310haa95532_0
prompt_toolkit            3.0.43               hd3eb1b0_0
psutil                    5.9.0           py310h2bbff1b_0
pubnub                    7.3.1                    pypi_0    pypi
pure_eval                 0.2.2              pyhd3eb1b0_0
pycparser                 2.21                     pypi_0    pypi
pycryptodomex             3.19.0                   pypi_0    pypi
pygments                  2.15.1          py310haa95532_1
pynwb                     2.5.0                    pypi_0    pypi
pyparsing                 3.0.9           py310haa95532_0
pyqt                      5.15.10         py310hd77b12b_0
pyqt5-sip                 12.13.0         py310h2bbff1b_0
pyqtgraph                 0.13.3                   pypi_0    pypi
pyside6                   6.5.3                    pypi_0    pypi
pyside6-addons            6.5.3                    pypi_0    pypi
pyside6-essentials        6.5.3                    pypi_0    pypi
python                    3.10.13              he1021f5_0
python-dateutil           2.8.2              pyhd3eb1b0_0
python-tzdata             2023.3             pyhd3eb1b0_0
python_abi                3.10                    2_cp310    conda-forge
pytz                      2023.3.post1    py310haa95532_0
pywin32                   305             py310h2bbff1b_0
pyyaml                    6.0.1                    pypi_0    pypi
pyzmq                     25.1.0          py310hd77b12b_0
qt-main                   5.15.2              h19c9488_10
qt-webengine              5.15.9               h5bd16bc_7
qtwebkit                  5.212                h2bbfb41_5
quantities                0.14.1                   pypi_0    pypi
referencing               0.30.2                   pypi_0    pypi
requests                  2.31.0                   pypi_0    pypi
rpds-py                   0.12.0                   pypi_0    pypi
ruamel-yaml               0.18.5                   pypi_0    pypi
ruamel-yaml-clib          0.2.8                    pypi_0    pypi
scikit-learn              1.3.0           py310h4ed8f06_0
scipy                     1.11.3          py310h309d312_0
setuptools                68.2.2          py310haa95532_0
shiboken6                 6.5.3                    pypi_0    pypi
simplejson                3.19.2                   pypi_0    pypi
sip                       6.7.12          py310hd77b12b_0
six                       1.16.0             pyhd3eb1b0_1
sortedcontainers          2.4.0                    pypi_0    pypi
sortingview               0.12.0                   pypi_0    pypi
spikecomparison           0.3.3                    pypi_0    pypi
spikeextractors           0.9.11                   pypi_0    pypi
spikefeatures             0.1.2                    pypi_0    pypi
spikeinterface            0.99.0.dev0              pypi_0    pypi
spikeinterface-gui        0.7.0                    pypi_0    pypi
spikemetrics              0.2.4                    pypi_0    pypi
spikesorters              0.4.5                    pypi_0    pypi
spiketoolkit              0.7.7                    pypi_0    pypi
spikewidgets              0.5.4                    pypi_0    pypi
sqlite                    3.41.2               h2bbff1b_0
stack_data                0.2.0              pyhd3eb1b0_0
tbb                       2021.8.0             h59b6b97_0
tblib                     3.0.0                    pypi_0    pypi
threadpoolctl             2.2.0              pyh0d69192_0
tk                        8.6.12               h2bbff1b_0
toml                      0.10.2             pyhd3eb1b0_0
tomli                     2.0.1           py310haa95532_0
toolz                     0.12.0                   pypi_0    pypi
tornado                   6.3.3           py310h2bbff1b_0
tqdm                      4.66.1                   pypi_0    pypi
traitlets                 5.7.1           py310haa95532_0
typing-extensions         4.8.0                hd8ed1ab_0    conda-forge
typing_extensions         4.8.0              pyha770c72_0    conda-forge
tzdata                    2023d                h04d1e81_0
urllib3                   2.0.6                    pypi_0    pypi
vc                        14.2                 h21ff451_1
vs2015_runtime            14.27.29016          h5e58377_2
wcwidth                   0.2.5              pyhd3eb1b0_0
wheel                     0.41.2          py310haa95532_0
widgetsnbextension        4.0.9                    pypi_0    pypi
xarray                    2023.9.0                 pypi_0    pypi
xz                        5.4.5                h8cc25b3_0
yarl                      1.9.2                    pypi_0    pypi
zarr                      2.16.1                   pypi_0    pypi
zeromq                    4.3.4                hd77b12b_0
zict                      3.0.0                    pypi_0    pypi
zipp                      3.17.0             pyhd8ed1ab_0    conda-forge
zlib                      1.2.13               h8cc25b3_0
zstd                      1.5.5                hd43e919_0
zm711 commented 10 months ago

That's weird because the numba and numpy are both still the same version in your environment... You could try to update threadpoolctl with pip instead 

pip install -U threadpoolctl

but if it keeps giving you the numba/numpy error I would consider downgrading numpy to 1.24. 

pip install numpy =1.24
litzj01 commented 10 months ago

Updating threadpoolctl worked, but it didn't fix the issue with numpy so I did have to downgrade it (I downgraded to numpy 1.25.0). That fixed the issue with initializing spike interface. I am trying to export the waveforms again which seems to be working. I'll keep you updated on how it goes. Thanks for your help!

zm711 commented 10 months ago

@litzj01, did this fix your issue? Just wanted to make sure we had solved the problem.

litzj01 commented 10 months ago

Yes, this seemed to have fixed the issue! Thanks for your help! Sorry for the late reply, I was without power for the week due to bad weather.

zm711 commented 10 months ago

Thanks for the response! Helps us keep track of things. And sorry about the power, hopefully the weather (and the waveform extracting) get a bit better!