search

AllenNeuralDynamics / aind-ephys-pipeline

Code Ocean pipeline for ephys processing with Kilosort2.5
MIT License
5 stars 2 forks source link

Concatenate recordings mode still broken #15

Open bjhardcastle opened 3 months ago

bjhardcastle commented 3 months ago

[continued from https://github.com/AllenNeuralDynamics/aind-ephys-preprocessing/issues/9]

Some checks in spikeinterface still appear to be incompatible with spikes from multiple recordings:

truncated output:

...

  POSTPROCESSING
  Found 1 json configurations
    Processing experiment1_Record Node 103#Neuropix-PXI-100.ProbeA-AP_recording
  Loading recording from binary JSON
    Number of original units: 318 -- Number of units after minimum spikes: 318
      Extracting raw waveforms for deduplication
  Traceback (most recent call last):
    File "/tmp/nxf.3YDP1Mlxfq/capsule/code/run_capsule.py", line 215, in <module>
      we_raw = si.extract_waveforms(
    File "/opt/conda/lib/python3.9/site-packages/spikeinterface/core/waveform_extractor.py", line 1672, in extract_waveforms
      we = WaveformExtractor.create(
    File "/opt/conda/lib/python3.9/site-packages/spikeinterface/core/waveform_extractor.py", line 271, in create
      raise ValueError(
  ValueError: The sorting object has spikes exceeding the recording duration. You have to remove those spikes with the `spikeinterface.curation.remove_excess_spikes()` function

...

https://codeocean.allenneuraldynamics.org/data-assets/64dbcd9c-148a-4d6c-aba7-2bd54438eb96/ecephys_712141_2024-06-11_11-50-06_sorted_2024-07-19_01-52-51

bjhardcastle commented 1 month ago

obs_intervals should also be included in the NWB units table to mark the sections between recordings (otherwise units will appear to have zero firing rate): https://pynwb.readthedocs.io/en/stable/pynwb.misc.html#pynwb.misc.Units