Mapping the parameter space in EEG/MEG analyses

[johalgermissen]

Project info

Title: Mapping the parameter space in EEG/MEG analyses

Project lead: Johannes Algermissen, @johalgermissen Looking for co-leads, both for the EMEA hub and also the other hubs!

Timezone: Berlin, Brussels, Cairo, Johannesburg UTC+2

Hub: Europe, Middle East and Africa. Looking for co-leads, both for the EMEA hub and also the other hubs!

Description: There is a complete djungle of ways to analyze EEG/MEG data. Some people have started to systematically map the impact of parameters in pre-processing and analyses of fMRI data (e.g. Botvinik-Nezer et al., 2020 a.k.a. "the NARPS project"; Bowring, Maumet, & Nichols, 2019; Carp, 2012). However, such an enterprise might be considerable more difficult for EEG/ MEG analyses with their many steps and the many existing software packages. There are some starting points on single pre-processing steps (e.g. Jaswal et al., 2020; Luck & Gaspelin, 2017, Robbins et al., 2020), but no attempt to systematically track the impact of all steps (like the NARPS project did).

One problem might be that there is not even yet a good common framework to compare different analysis pipelines with each other. There are the COBIDAS-MEEG guidelines (still a pre-print), which are a very good starting point, but which were designed as a detailed checklist for reporting single isolated analyses rather than a framework for identifying commonalities between different analyses. Furthermore, the fact that it is very detailed and exhaustive does not make it very user-friendly (yet).

The aim of this hackathon is to create such a framework in a more bottom-up way, focusing on the essential steps needed to compare different pipelines. We will start by looking at existing pre-registrations, tracking what people do and what they struggle with, to then come up with a framework which we map onto software options, and finally maybe start with an adaptive app that facilitates pre-registration of EEG/MEG analyses.

Such an enterprise might be very fruitful for subsequent empirical exploration of how the parameters involved in EEG/MEG analyses matter for the results, e.g. in the EEGManyLabs project or the ManyPipelines project. Also, it might facilitate the creation of a user-friendly EEG/MEG pre-registration templates (currently in the making, see e.g. this one started at the SIPS meeting 2019, which will also be worked on during Thursday June 18 in the afternoon at a virtual hackathon at the MPI-CBS, and this spin-off started at the Donders Institute in November 2019).

Link to project:

• GoogleDocs folder (click and request access, I will add you)
  for real-time collaboration on text documents (word files, spreadsheets)
• Github repository
  for working on more permanent documents that need version control, as well as code (if we start building an app in the end)

Mattermost handle:

Goals for the OHBM Brainhack:

• Find and code available pre-registrations/ registered reports using EEG/MEG data
• Defining a framework of essential steps useful to compare different analyses
• Map different software options onto these steps
• (Start) designing an ("automated"/ "adaptive") EEG/MEG pre-registration template (app people can fill in, with help links etc.)

Good first issues:

• Find repositories/ webpages that track neuroimaging projects/ papers with pre-registrations
• Fill in those projects/ papers in a huge list (example)
• Rate submitted projects/ papers on several items (e.g. software used, pre-registered processing and analysis steps, deviations from the pre-registration reported in the paper)

On Thursday (June 18) afternoon, there will be a parallel hackathon organized by @mariellapaul and @aschetti via the Max-Planck-Institute for Human Cognitive and Brain Sciences (Leipzig, Germany) that will aim to make progress on this pre-registration template. For more information see here.

Skills (at least one of the following, this is an AND/OR list):

• Ability to find pre-registrations on the internet ("detective hunt")
• Ability to quickly find and extract the relevant information in papers/ pre-registration documents (a bit similar to "repliCats")
• (Some) experience with EEG/ MEG experiences (e.g. one of the following software packages: BESA, BrainStorm, BrainVisionAnalyzer, EEGLAB, Fieldtrip, MNE, Nutmeg, SPM)
• Access to one/ several EEG/MEG analysis software packages during the conference (see previous bullet point; the more "exotic", the better)
• Experience with pre-registration
• For the template: Experience with programming a simple app with a decision-tree in the background (javascript, html?)

Participant number: 25 (max.). This project is very well suited to work on separate sub-tasks without interfering with each other. It will be easy to form different Discord video channels to work in small sub-groups on one specific task or within one separate document. It is also suited for joining only one sub-task for a few hours. It is possible to start one subtask in one time zone hub and finish it in another.

Chat channel: Mattermost brainhack platform, public channel hbmhack-mapMEEG

Video channel: Discord server with several text and voice channels, link will on the Mostmost channel [hbmhack-mapMEEG]

Twitter-size summary: Mapping the parameter space in EEG/MEG analyses https://github.com/ohbm/hackathon2020/issues/139 @johalgermissen The parameter space of M/EEG pipelines is like a jungle. Let's map it! Identify pipelines, set up a framework, map software on it, create a template.

OHBMHackathon #Brainhack #OHBM2020

Project snippet for the OHBM Brainhack website title: "Mapping the parameter space in EEG/MEG analyses" image: mapMEEG_Logo.png details: "PROJECT DESCRIPTION.
The parameter space of M/EEG analysis pipelines is like a jungle. We will map it! We will first identify and classify available pipelines (based on available pre-registrations) and identify common challenges researchers face, then come up with a semantic framework to create and compare different pipelines, map software options onto the different analysis steps, and finally start designing an adaptive app for M/EEG pre-registration!
More information in this github issue

Project submission

Submission checklist

Once the issue is submitted, please check items in this list as you add under 'Additional project info'

Please include the following above (all required):

• [x] Link to your project: could be a code repository, a shared document, etc. See here
• [x] Include your Mattermost handle]. If you do not have an account, please sign up here.
• [x] Goals for the OHBM Brainhack: describe what you want to achieve during this brainhack. See here.
• [x] Flesh out at least 2 "good first issues": those are tasks that do not require any prior knowledge about your project, could be defined as issues in a GitHub repository, or in a shared document, cf here.
• [x] Skills: list skills that would be particularly suitable for your project. We ask you to include at least one non-coding skill, cf. here.
• [x] Chat channel: A link to a chat channel that will be used during the OHBM Brainhack. This can be an existing channel or a new one. We recommend using the Brainhack space on mattermost, cf. here.
• [x] Video channel: Please create a video channel that will be used during the OHBM Brainhack and share it in your chat channel above. This can be an existing channel or a new one. For instance a jitsi meet room, cf. here.

You can also include information about (all optional):

• [x] Number of participants, cf. here
• [x] Twitter-size summary of your project pitch, cf. here
• [x] Provide an image of your project for the OHBM brainhack website
• [ ] Set up a kanban board on your repository to better divide the work and keep track of things, cf here
• [x] Project snippet for the OHBM Brainhack website, cf. here

We would like to think about how you will credit and onboard new members to your project. We recommend reading references from this section. If you'd like to share your thoughts with future project participants, you can include information about (recommended):

• [x] Specify how will you acknowledge contributions (e.g. listing members on a contributing page).
• [x] Provide links to onboarding documents if you have some.

[johalgermissen]

hi @ohbm/project-monitors : My project is ready! (will add a picture and text for the brainhack webpage later today)

[johalgermissen]

Filed pull request to add picture and adapted text o brainhack webpage

[Remi-Gau]

Filed pull request to add picture and adapted text o brainhack webpage

hey @johalgermissen Thanks for that. I have just sent you a PR on top of that to fix a small indent issue in the yml file. If you merge it it should update this PR.

[Remi-Gau]

I will be following this project for the @ohbm/hack2020-social_media

[johalgermissen]

Okay thanks great. I think the project description is basically read now. Will look into the kanban board and into how acknowledgments work tomorrow. Will also populate the google docs folder further tomorrow.

[complexbrains]

@johalgermissen You indicated that you will share the video link tomorrow, so may I ask you to share your video link, when it is ready, only on your Mattermost channel please to avoid any unwanted interferences, and please pin the message to your channel, please? We look forward to meeting you tomorrow! 🤗

[johalgermissen]

@complexbrains yes, already saw the alert on Mattermost! Now also updated the project description here.

[complexbrains]

Thank you! Then I will complete the review of your project!

[complexbrains]

Dear @johalgermissen Thank you very much for your project submission. Your project looks ready. Welcome aboard! 🤗 🎊

[carolyog]

Hi @johalgermissen! I found your project very interesting and I'd like to take part. However, I belong to the Americas hub (UTC-3, São Paulo, Brazil more precisely). so I'll be attending the Americas hub Traintrack. Would it be a problem? Thank you!