Loading external (MNE-Epochs) data into fitgrid

[AJQuinn]

Hello - I'm having trouble loading some data into fitgrid via the dataframe option. I have some epoched data in MNE-python Epochs format which I've transformed to a pandas data frame and am trying to load into fitgrid with

ep = mne.read_epochs('/path/to/my/meg_data_epochs.fif')
ep.pick_types(meg='grad')

df = D.to_data_frame()
epochs = fitgrid.epochs_from_dataframe(df, 'time', 'condition', ep.info['ch_names'])

The dataframe seems to contain the expected information, including the time and conditions columns

       time condition  epoch    MEG0112    MEG0113    MEG0122    MEG0123 
0      -200   deviant      0  50.459915  52.066864  50.591106  51.209538  
1      -198   deviant      0  22.952164  68.193894  15.114554  25.454615  
2      -195   deviant      0 -37.157683  88.652950  72.027299 -49.471064 
3      -192   deviant      0 -28.262617  81.412390  77.451656  11.326285  
4      -190   deviant      0  -3.821066  57.508205  54.739573   1.642245
...................................
88229   490      rep3    458  57.493560  10.382781 -15.607079 -10.872845 
88230   492      rep3    458  47.027466  37.594036  11.246346  -0.899396 
88231   495      rep3    458  30.517312  29.242046  39.432882 -15.305428 
88232   498      rep3    458  10.922786  22.973594  51.378524  39.577536 
88233   500      rep3    458  13.658708  -9.010685  36.900157  53.443320

However, I'm getting the following traceback from fitgrid.epochs_from_dataframe

~/src/fitgrid/fitgrid/epochs.py in __init__(self, epochs_table, time, epoch_id, channels)
     50             if item not in epochs_table.index.names:
     51                 raise FitGridError(
---> 52                     f'{item} must be a column in the epochs table index.'
     53                 )
     54

FitGridError: condition must be a column in the epochs table index.

Not sure what I've got wrong, perhaps the dataframe is not properly formatted? I'm running fitgrid version 0.5.0 in python 3.7.

Cheers

[turbach]

@AJQuinn thanks this is a good use case. I have fed MNE epochs to fitgrid along these lines with a modicum of dataframe wrangling.

Far as I can see, there's not much wrong, your df dataframe looks fine.

fitgrid wants to slurp a dataframe already row-indexed for time and epoch for efficient slicing.

When MNE dumps mne.Epochs D to the pandas.DataFrame df it puts index-like information in the df["epoch", "time"] columns but not in the index itself which is the default row index 0 .. 88233 as you see on the left.

So I think one line of set_index() to convert the columns to a (Multi)Index should make fitgrid happy except swap epoch for condition which I'm guessing is a categorical variable not a single-trial epoch index.

df = D.to_data_frame()
df.set_index(['time', 'epoch'], inplace=True)
epochs = fitgrid.epochs_from_dataframe(df, 'time', 'epoch', ep.info['ch_names'])

This is along the lines of the v0.5.0 Quickstart step 2. which also starts with a row-indexed dataframe and the time and epoch info in the columns and does the set_index() trick.

p3_df.set_index(['epoch_id', 'time_ms'], inplace=True)
p3_epochs_fg = fg.epochs_from_dataframe(
    p3_df,
    epoch_id='epoch_id',
    time='time_ms',
    channels=['MiPf', 'MiCe', 'MiPa', 'MiOc'],
)

[AJQuinn]

Great, this is working. Thank you!