Question: multiple events per trial

[kingjr]

Hello, I'm trying to import some data from fieldtrip to MNE-Python.

However, my data contains multiple events per trial: e.g. (visual or auditory) x (left or right) x (relevant or distractor).

What do you think is the best way to keep these various information in the events, and how can I then some interaction (e.g. visual x left x relevant; e.g. visual left OR auditory right)?

Thanks!

[dengemann]

@kingjr do you say that this is not multiple categories per event or different events in one trial?

(visual or auditory) x (left or right) x (relevant or distractor)

could read like one event that is visual + left + relevant, and so on, just single cells from a 2 x 2 x 2 design. If that's the case you just need sufficient event codes, 6 different codes for example. If you say you have to deal with let's say 3 different, individual events in one trial, it's gonna be complicated with the existing functions.

[kingjr]

no it's three categories of events per trial: in one trial, you only have one event with multiple features.

I have up to 10 categories of events per trial, which is a bit of a pain to combine and then parse but ok. I'll go for summing them up... Could be easier to have multidimensional arrays of events like in fieldtrip?

[larsoner]

One way to keep things simple(ish) is to use different decimal places to encode the type. For example, the ones digit could be relevant or distractor as 1 or 2, tens could be LR could (10 or 20), and hundreds could be vis/aud (100 or 200). Then seeing / coding a trial as 211 you know it's a visual left relevant trial. But this is really up to you...

I'm not really sure what the multidimensional array would gain us. One problem with adding that would be that, historically, event numbers only have a single dimension. This means that the file formats -eve.lst and -eve.fif are only designed to handle this case. So it's unlikely we'd go to multidimensional arrays in the near future, especially if there are equivalent not-too-bad ways of doing things with a simpler single-dimensional solution.

[dengemann]

+1 on what Eric said.

[dengemann]

Use a good event coding scheme and collapse acros factors as needed. Dictionaries also help.

[kingjr]

@Eric89GXL yes that's what I was thinking of doing. @dengemann : What do you mean by using dictionaries here?

[dgwakeman]

Hi, the best way to deal with this in my opinion is to use the binary nature of the triggers. I have done this in my own experiments, so if I have say three different event types: famous faces, scrambled faces, and unfamiliar faces, with three different subtypes (first presentation, second presentation, third presentation). I can modulate these differently. I can use 1,2,3 to represent the presentation subtype and the 4, 8, 16 to represent the image type. Thus my events become: 5, 6, 7 for famous faces (first, second, and third presentation). 9, 10, 11 for scrambled, and 17 18 19 for unfamiliar faces.

Furthermore, I know at least the C-code has ways of intelligently combining these (with the --ignore option). I'm not sure if this has been added to mne-python (I haven't gotten comfortable yet with the way it deals with averaging/epochs).

[kingjr]

@dgwakeman: Yes, this is close to Eric's proposal, isn't it?

In any case, from what I understand, these are ad-hoc tricks. Note that one may want to code for continuous values (e.g. the angle of a Gabor, and multicategorial etc). I was thus wondering whether MNE-Python had a flexible and efficient way of dealing with this variability of event coding.

[agramfort]

any suggestion of API is welcome. Basically if you had to propose a pythonic way of doing this boolean logic of conditions how would like to write it?

[dgwakeman]

@kingjr Yes, it is similar, although there is already code for dealing with this (at least in the C-code). I did not use it as an ad-hoc trick at all. I created my stimulus program to send these triggers to the machine, so they are built in already. I am currently trying to come up with a better strategy for implementing averaging (it looks like, I may have to mess with epoching as well, but I haven't had time to finish the other Issues I have on github, so I should get those done now).

[mainakjas]

I feel that there should be some way of combining epochs in a list of epochs. E.g.,

epochs.combine_trials([['visual', 'auditory'], ['left', 'right'], ['relevant', 'distractor']])

and this will either modify inplace or return a list of epochs with 3 items instead of 6. Does this address the use case intended?

[agramfort]

assuming event_id is explicit epochs['visual*'] would return all visual trials left or right?

[mainakjas]

So, the event dictionary would have keys visual-left-relevant, visual-right-relevant and so on? And the user could do epochs[visual*] ? That would be super-neat in my opinion.

[agramfort]

yes that's my idea

[dengemann]

how would that work for cases where you want to collapse one dimension but not the others. For example ignore relevance?

[mainakjas]

If we allow regular expressions, you can possibly do any kind of string operations such as ignore relevance?

[agramfort]

yes regexp can do this.

[dengemann]

but then you need a strict delimiter structure. The semantics is not clear otherwise. In a fully factorial design it would mean: collapse along the dimension that is not selected by the regexp. But what if you have strings that don't match:

visual-left-relevant, visual-right-elephant

or

visual-left-relevant, visual_right:elephant

[mainakjas]

basically, we allow regexp support and the rest is up to the user to exploit using right choice of strings. no?

I'd be happy to see the collapsibility extended to other functions like plot_events and its legends etc.

[dengemann]

basically, we allow regexp support and the rest is up to the user to exploit using right choice of strings. no?

but collapsing won't work if the namings are not symmetric. If some data is e.g. not fully factorial we need to guess that and throw errors / warnings.

[agramfort]

you seem to anticipate more than me the problem. Can you be more explicit?

[dengemann]

@agramfort we need to properly engineer this, otherwise collapsing rules will be magic and it will be impossible to predict how many epochs and which ones will be put in a new label. If we unpack this a bit it means that what is not matched by the regexps will be collapsed. So if you would pass a regexp that for examples leads to collapsing left and right, the new epochs are expected to have the following keys (which leads to another point, auto renaming of keys ... and auto-resetting trigger vlaues):

visual-relevant, visual-irrelevant, auditory-releveant, auditory-irrelevant

Stuff like this will work predictably as long as the naming is clean. otherwise it's gonna be messy.

This stuff somewhat reminds me of the pandas.groupy object. There you can collaps by factor:

df_group = data_frame.groupby([['modality','relevance']])

This would lead to collapse direction. But in the pandas case it's easier because you have column labels (factors) and row entries as factor levels. To imiate this we would at least need some sort of factorial indexing which relates certain triggers to a factor. But that would be easier on the long run, I think.

Putting all this in __getitem__ might be too much. Maybe a groupy / subset function or a method would be cleaner. It could then also take arguments regarding key renaming and value resetting, etc.

[dengemann]

Or maybe we need to add two things. Regexp without collapsing, just get what is matched + additional functionality to easily handle collapsing along one or multiple dimensions of the design.

[agramfort]

I fear we need to sit down together to converge on this...

[kingjr]

I agree with @dengemann that we need to account for non fully-factorial design. Also, note that not all designs are categorical. For instance, one may code for the contrast of a stimulus, the angle of a bar, the pitch of a sound etc, so, ideally, one could give a name, a number, or a Boolean as a code.

At the moment, I am using a separate object coding for events and checking this conditions of interest in a small loop e.g. selection = [((e['modality']=='visual' or e['modality']=='auditory') and e['relevant']==True) for e in events] epochs[selection]

So ideally, this separate object would actually be integrated within the epochs.

[agramfort]

So ideally, this separate object would actually be integrated within the epochs.

can you show the code of this object? I understand the examples but I miss the big picture to think of the best API.

[mainakjas]

hmm ... if we go for regexp without collapsing -- that is simply return all trials that match the criteria, wouldn't that do the trick? Numbers etc can all be coded in the dictionary keys. Why do we actually need to have new methods for collapsing because we already have combine_event_ids that does that, no?

[dengemann]

yes, but then you could also argue not to add anything new. You can do all this by hand, using dicts, dict comprehension and the functions we have. it wll only take you a couple of lines. the question is if we can simplify it

[agramfort]

ok suggestion. Let's say event_id can be

event_id = {'left/auditory': 1, 'right/auditory':2, 'left/visual':3, 'right/visual':4}

and epochs['visual'] returns all visual epochs?

[agramfort]

or maybe more readable

event_id = {'leftauditory': 1, 'rightauditory':2, 'leftvisual':3, 'rightvisual':4}

[jona-sassenhagen]

From a linguistics perspective and in a regression context, I agree with @kingjr continuous predictors would be very cool.

[kingjr]

If that's of any help, with @dengemann, we now use panda.DataFrame so as to easily deal with multiple discrete and continuous conditions.

[agramfort]

can you be more explicit how you do this within epochs in mne?

[kingjr]

I do it outside..

events = pd.DataFrame({cond1='foo', cond2=0.10}, {cond1='bar', cond2=3.})
selection = np.where(events['cond1']=='foo' & events['cond2']>1)[0]
epochs[selection].average()```

[agramfort]

if it's 3 lines of code why making the MNE code more complex???

[dengemann]

if it's 3 lines of code why making the MNE code more complex???

+1

[kingjr]

if it's 3 lines of code why making the MNE code more complex??? I agree. The only minus is that keeping the two objects separated increases the risk of breaking their link.

# sub selection of epochs
sel = events[modality=='visual']
epochs_vis = all_epochs[sel]
# keep cond2 in events for further processing
epochs_vis.events[:, 2] = events['left_right'][sel]   # whoo, careful what you do here boy

as opposed to

epochs_vis = all_epochs[sel]
print epochs_vis.events['left_right']  # Ooh yeah! neat

But obviously, changing the mne.events into a pd.dataframe is rather expensive, so let's forget about this.

[agramfort]

nobody thinks my:

event_id = {'leftauditory': 1, 'rightauditory':2, 'leftvisual':3, 'rightvisual':4} epochs['visual']

would solve a practical problem?

[mainakjas]

I'm +1 on this

[larsoner]

It would be useful, yeah. I am had not been pushing for it because I have built a system to take care of this in my own analysis pipelines already, as I imagine others have, too. That being said, thinking about it now, it would be really nice to just get rid of that code and would probably simplify things at my end, actually, so +1 from me, too. If we got it in for 0.9 we could get feedback on it, too. Anybody have the bandwidth to implement it? If not, I might be able to squeeze it in next week.

[larsoner]

Why use __ instead of /, just because you expect people might have used / but not __? We could have a condition_separator argument that defaults to / that people could change -- I think / is a more natural choice. It naturally lends itself to the interpretation "or" or "hierarchy" (like a filesystem), which is nice.

[agramfort]

No strong feeling. __ is used in django and scikit-learn but / is the Eeglab way apparently.

[jona-sassenhagen]

It's a really new EEGLAB thing though, I doubt many people would expect it. Though I agree with @Eric89GXL 's reasoning - / looks like hierarchy more.

[agramfort]

ok but when you do visual__left it's not a hierarchy but more multi labels.

[larsoner]

I think that / is what I'd expect to be the separator as a user, as it can actually be interpreted either way. In fact I'd probably do visual/left or /visual/left depending if I wanted to think of it as multi labels or hierarchy...

[agramfort]

Whatever on my side. As long as we document it.

[jona-sassenhagen]

Would this be as simple as inserting this

if (any(["/" in k for k in self.event_id.keys()]) 
       and key not in self.event_id.keys()):
    key = [k for k in list(self.event_id.keys()) if key in k.split("/")]

in Epoch's __getitem__ method? Or am I missing something?

[larsoner]

Pretty much, I was thinking something similar, plus a bunch of tests to make sure it actually works as intended :) If you have time (and no other PRs pending for release!), please implement. I guess you have https://github.com/mne-tools/mne-python/pull/1795 but you're not the blocker there.

[larsoner]

...also the seperator should be configurable via a property.

[agramfort]

I think we have 50% of the PR here ;) the rest is a test ;)