search

mne-tools / mne-python

MNE: Magnetoencephalography (MEG) and Electroencephalography (EEG) in Python
https://mne.tools
BSD 3-Clause "New" or "Revised" License
2.7k stars 1.31k forks source link

BUG: Gaps in neuralynx not handled properly #12247

Open larsoner opened 10 months ago

larsoner commented 10 months ago

The low-level read_neuralynx_ncs function detects the presence of gaps in the .ncs file and issues a warning.

It sounds like mne.io.read_raw_neuralynx() should minimally check for temporal gaps between neo segments and issue a warning? For example, it should check that each neo.Segment[i] object starts when the neo.Segment[i-1] ended and raise a warning if this is not the case (i.e. there's temporal gap, assuming the information in neo is accurate)? And potentially also reconstruct/fill/mark missing samples such that the time axis (i.e. raw.times) is continuous and valid.

If this is on track, happy to open a separate issue and work on this.

Originally posted by @KristijanArmeni in https://github.com/mne-tools/mne-python/issues/11969#issuecomment-1832082578

larsoner commented 10 months ago

Starting with a warning is good. Correct behavior would be (instead of warning) to fill these gaps with zeros and annotate with BAD_ACQ_SKIP, then MNE-Python functions will handle them properly and as you say the time axis will be correct. So if you want to jump straight to that and skip the warning business feel free!

Then the separate sample rate problem will be solved by #11408, which I came up with a tentative workable plan for.

scott-huberty commented 10 months ago

Starting with a warning is good. Correct behavior would be (instead of warning) to fill these gaps with zeros and annotate with BAD_ACQ_SKIP, then MNE-Python functions will handle them properly and as you say the time axis will be correct. So if you want to jump straight to that and skip the warning business feel free!

Eyelink systems sometimes also have tracker dropout between trials, resulting in gaps in those recordings. We filled these gaps with NaN and annotated with BAD_ACQ_SKIP (we used NaN becasue 0 technically is a real value for eyegaze data), but the presence of those NaN's does cause an issue for a lot of preprocessing functions that operate on the time axis and don't respect the BAD_ACQ_SKIP annotations, i.e. resampling.. I'll definitely be interested in seeing how you handle the gaps in this case so that maybe we can try to make read_raw_eyelink consistent with your design choice.

larsoner commented 10 months ago

@scott-huberty I hatched what I think is a workable plan for resampling in https://github.com/mne-tools/mne-python/pull/11408#issuecomment-1831037882 -- feel free to look there and let me know what you think