GSoC idea: Create realistic simulated EEG-based MI-BCI data with MNE-Python

[catalinamagalvan]

Dear MNE-Community, I’m Catalina Galván, a current PhD student from Argentina. For my analyzes, I've been working with my advisor @vpeterson in creating realistic MI-BCI data and I believe it would be interesting for the community to make it an MNE module. In addition, I think this could be a suitable idea for participating in the upcoming GSoC with MNE-Python as my organization. Here's a draft of the proposed idea, let me know what you think. Thank you!

Motivation Brain-Computer Interface (BCI) methods are commonly studied employing Electroencephalogram (EEG) data recorded in human experiments. Data recordings based upon some BCI paradigms, such as Motor-Imagery (MI), suffer from different sources of variability, which cannot be controlled during the experimental design and setup (subject motivation, fatigue, etc). Due to that fact, when testing hypotheses or newly developed methods, there is always a need to conduct simulations so as to ensure that the model assumptions are being accomplished, and thus, understand the strengths and weaknesses of the method. Although real experiments, and thus real data, will be always ultimately needed for testing and verification of experimental hypotheses, realistic simulation can be useful to construct a solid ground for answering such hypotheses and assumptions. Moreover, in some situations real data is either costly to generate or to access, narrowing the evolution of new or different algorithmic solutions. Furthermore, for such methods that require a large amount of data, as deep learning methods do, realistic simulated EEG-based BCI data can serve as a tool for raw data augmentation.

Goal+ The aim of this project is to facilitate EEG-based MI-BCI data generation employing the mne.simulation module. These data might be used to test different BCI classification methods, and thus, should ensure model assumptions or data augmentation and, ideally, be as similar as possible to real data. In this scope, realistic data based upon different sources of noise, artifacts and changes at the level of the subject will be constructed. The simulated experiment timelines would include different signal components:

• Task-dependent activity.
• Noise and artifacts activity.
• Cognitive effects (fatigue, feedback, attention, learning, etc).

In this context, each signal component is assumed to have three properties: where (dipoles of interest in the cortex), what (waveform), and when (events), which are the inputs for the add_data method of mne.simulation.SourceSimulator class. The idea is to generate suitable where, what, and when inputs for different types of possible MI-BCI scenarios, i.e., simulate “good” and “bad” subjects, simulate subject’s fatigue, simulate the impact on the EEG data due to feedback, and so on. The framework would be designed with the goal that all the parameters of interest could be specified by the user from a single high-level script. Examples of parameters are: BCI paradigm, number and duration of trials/repetitions, addition of noise/artifacts at a given time onset, addition of fatigue/feedback/attention/learning effects at a given time onset.

See https://github.com/mne-tools/mne-python/issues/5058, https://github.com/mne-tools/mne-python/issues/5924, and https://mne.tools/stable/auto_examples/simulation/plot_simulated_raw_data_using_subject_anatomy.html for some ideas.

[welcome[bot]]

Hello! 👋 Thanks for opening your first issue here! ❤️ We will try to get back to you soon. 🚴🏽‍♂️

[larsoner]

This sounds like a cool project! I think such simulations would have great value to the neuroscience community.

My first thought is that this sounds better suited to a methods paper/project than a GSoC project. In other words, it seems like what you propose is actually too novel from a neuroscience perspective to just implement in MNE. It seems like there should be a paper describing these methods of simulation, testing them, etc. The scientific review process would be good for settling on these issues.

Then once there are established algorithms, they could be included in MNE. I could be mistaken though, in that there could already be established algorithms / a "right way" to implement the what/when for these various noise sources.

[vpeterson]

Hi @larsoner! Indeed, this is a whole project itself (part of @catalinamagalvan PhD thesis), which we are currently working on. That's why, we thought that the GSoC could serve us (especially Catalina) to push out one piece of this project (the code, and its compatibility with MNE APIs). At the same time, we understand that making all simulations could be too much for the GSoC timeline. We can think about at least implementing one of such subject-based noise/artifacts, like fatigue (which effects in EEG have been widely studied, i.e. what/when/where is known). What do you think? Is this a suitable idea?

[drammock]

We can think about at least implementing one of such subject-based noise/artifacts, like fatigue

Since the GSoC deadline is coming soon, I think it makes sense to move forward with a smaller implementation plan as you suggest @vpeterson (i.e., stick to just the kinds of noise/artifacts that are well-established in the literature). I'll ping the rest of the core devs to see who is the best choice to be mentor for this project, and hopefully by Monday things will be falling into place.

[catalinamagalvan]

Thank you @drammock for your comments. After talking to @larsoner and also considering your recommendation, we believe that a good scope for the GSoC project is to bring the well-established in the literature simBCI framework for simulating BCI data from MATLAB to MNE-Python.

Taking into account the above, I've been working on my GSoC proposal doc, needed for the application, which is available here. I would very much appreciate if you could look at it and comment on it. Thanks!

[larsoner]

Closing since this was relevant to last year's GSoC