SpikeInterface / spikeinterface

A Python-based module for creating flexible and robust spike sorting pipelines.
https://spikeinterface.readthedocs.io
MIT License
527 stars 187 forks source link

Add CI for ingesting neuroscope spiking data #520

Closed bendichter closed 2 years ago

bendichter commented 2 years ago

https://gin.g-node.org/NeuralEnsemble/ephy_testing_data/src/master/neuroscope/dataset_1

samuelgarcia commented 2 years ago

@bendichter @CodyCBakerPhD I tried to add this to SI tests. The reader we have in Si in only for recording.

In the (old) neo implmentation, the dataset is supposed to contain a ".dat" file. Here there is a ".eeg" file instead. The xml do not related any eeg file except in the help section. Do you have a reference information about this format and what are the extension for "coninuous" (dat/eeg/lfp/xxx) ? We need to make a patch on neo side.

samuelgarcia commented 2 years ago

I find it. http://neurosuite.sourceforge.net/formats.html The official format page is mentioning .dat, .lfp, .fil as possible extensions for continuous traces. The archive of neuroscope/dataset_1 have a eeg I am not sure this eeg is part of the official format but looks a home made preprocessing (filetering). Do you have info why they change lfp to eeg ? Any way the neo patch should explore some extension for the continuous dat/lfp/eeg/fil

CodyCBakerPhD commented 2 years ago

All the working code for Neuroscope types can be found in the LEGACY spikeextractors: https://github.com/SpikeInterface/spikeextractors/blob/master/spikeextractors/extractors/neuroscopeextractors/neuroscopeextractors.py

.lfp files are read in an equivalent manner to the .dat (same binary structure) but they do have separate metadata (namely the different sampling frequency) in the .xml

.lfp files are equivalent to .eeg in every way. Don't know exactly when or why, or if it's required or just convention.

I've never seen a .fil file over the course of the past ~10 conversions.

samuelgarcia commented 2 years ago

OK thanks. I will add this in neo soon.

bendichter commented 2 years ago

@samuelgarcia since this is for spike sorting data specifically, I think .eeg can be safely ignored here