mne-tools / mne-python

MNE: Magnetoencephalography (MEG) and Electroencephalography (EEG) in Python
https://mne.tools
BSD 3-Clause "New" or "Revised" License
2.7k stars 1.31k forks source link

Creating epochs is dropping regardless of rejection/flat setting to None #10055

Closed mscheltienne closed 2 years ago

mscheltienne commented 2 years ago

A bit of an odd case here, I don't know if it's a bug or intended and badly explained, but when working on a dataset and creating 4 equal epochs manually (should have used make_fix_length_epochs), 3 epochs out of 4 were dropped. When replicating with the sample dataset, all 4 are dropped, and I don't get why.

import math
from pathlib import Path

import mne
import numpy as np

mne.set_log_level('DEBUG')

directory = Path(mne.datasets.sample.data_path()) / 'MEG' / 'sample'
fname = directory / 'sample_audvis_filt-0-40_raw.fif'

# load and crop Raw
raw = mne.io.read_raw_fif(fname, preload=True)
raw.pick_types(eeg=True)
raw.del_proj('all')
raw.crop(10, 25.98)  # shape (59, 2400)

# create Epochs
events = np.zeros((4, 3))
for k in range(4):
    events[k, 0] = math.floor(raw.info['sfreq'] * 4 * k)
events[:, -1] = 1
events = events.astype(np.int16)

epochs = mne.Epochs(raw, events, event_id=1, tmin=0, tmax=3.99, baseline=None, 
                    picks='eeg', reject=None, flat=None, preload=True)

I'm getting this logging output:

Loading data for 4 events and 600 original time points ...
    Getting epoch for -7952--7352
    Getting epoch for -7352--6752
    Getting epoch for -6751--6151
    Getting epoch for -6151--5551
4 bad epochs dropped

Any idea on what I overlooked before I start digging?

larsoner commented 2 years ago

Look at epochs.drop_log, it should say why. NO_DATA means it couldn't get data for the entire range (tmin or tmax extends outside the raw range)

mscheltienne commented 2 years ago

Ok, so indeed I get (('NO_DATA',), ('NO_DATA',), ('NO_DATA',), ('NO_DATA',)), but epochs with tmin=0, tmax=3.99 on a raw of 16 seconds should work no?

larsoner commented 2 years ago
raw.crop(10, 25.98)  # shape (59, 2400)

Your data now start at whatever raw.first_time was before the crop, plus another 10 seconds. If you change your event creation to this it should work:

events[k, 0] = math.floor(raw.info['sfreq'] * 4 * k) + raw.first_samp

Or even better, use mne.make_fixed_length_events

mscheltienne commented 2 years ago

Ok, thanks a lot @larsoner I'm getting easily confused with this system of first_time.. I get now that I am not supposed to create this event array manually..

I proposed a quick PR to improve a bit the docstring for events across the MNE functions.