FLOW - Integrating EEG analysis scripts in a cloud-based database management system

ephathaway commented 3 years ago

Project info

Title: FLOW - Integrating EEG analysis scripts in a cloud-based database management system

Project lead: Evan Hathaway, [Ian Brown] (https://github.com/ianbrown9475)

Timezone: Los Angeles UTC-7

Description:

Brain Electrophysiology Laboratory is proud to present FLOW (Forward Looking Operations Workflow), an all-in-one solution for EEG database management, ERP analysis pipelines, and visualization with AWS cloud or on-premises server implementations. We use containerization technology to run ERP analysis Python scripts as modular units with I/O to the FLOW database, thus facilitating reproducibility across different stages of analysis. The containerization approach allows easy integration of existing analysis scripts, thus making FLOW an environment that can be adjusted to lab needs.

BEL aims to provide the community with the means to write their own EEG analysis workflows and share them with others. We are working to create an environment where researchers can pilot workflows in a live Jupyter Notebooks integration of FLOW, then convert these to streamlined FLOW workflows which can be shared with the community. Our project will help guide participants through creating, packaging, and containerizing analysis scripts using open source Python analysis tools and Docker. We will then demonstrate how these workflows can be integrated in FLOW for data management, analysis, and visualization through our Sourcerer software. We will do this via a live instance of FLOW that will be up throughout the course of the hackathon. We invite interested parties to try out FLOW for themselves in order to see what this software has to offer.

Link to project:

https://github.com/BEL-Public/eeg-workflows
Our live FLOW instance (Contact project leads to create an account)

Mattermost handle: @ephathaway, @ianbrown9475

Goals for the OHBM Brainhack

Integrate an EEG analysis workflow from an outside collaborator into FLOW

Good first issues:

https://github.com/BEL-Public/eeg-workflows/issues/4#: Take a look at our contribute section to learn how to add your own EEG analysis workflow to FLOW
https://github.com/BEL-Public/eeg-workflows/issues/5#: Improve the EEG artifact detection module by implementing detection of specific artifact types

Skills:

EEG/ERP analysis
Python (beginner)
Jupyter Notebooks
Docker (helpful, but not required)
Git - 2

Chat channel: https://mattermost.brainhack.org/brainhack/channels/brain-electrophysiology-laboratory---flow

Image for the OHBM brainhack website

Project submission

Submission checklist

Once the issue is submitted, please check items in this list as you add under 'Additional project info'

Please include the following above (all required):

[x] Link to your project: could be a code repository, a shared document, etc. See here
[x] Include your Mattermost handle (i.e. your username). If you do not have an account, please sign up here.
[x] Goals for the OHBM Brainhack: describe what you want to achieve during this brainhack. See here.
[x] Flesh out at least 2 "good first issues": those are tasks that do not require any prior knowledge about your project, could be defined as issues in a GitHub repository, or in a shared document, cf here.
[x] Skills: list skills that would be particularly suitable for your project. We ask you to include at least one non-coding skill, cf. here.
[x] Chat channel: A link to a chat channel that will be used during the OHBM Brainhack. This can be an existing channel or a new one. We recommend using the Brainhack space on mattermost, cf. here.
[x] Provide an image of your project for the OHBM brainhack website

You can also include information about (all optional):

[ ] Someone co-leading the project in the timeslot you have not selected to provide additional visibility.
[ ] Number of participants, cf. here
[ ] Twitter-size summary of your project pitch, cf. here
[ ] Set up a kanban board on your repository to better divide the work and keep track of things, cf here
[ ] Project snippet for the OHBM Brainhack website, cf. here

We would like to think about how you will credit and onboard new members to your project. We recommend reading references from this section. If you'd like to share your thoughts with future project participants, you can include information about (recommended):

[ ] Specify how will you acknowledge contributions (e.g. listing members on a contributing page).
[ ] Provide links to onboarding documents if you have some.

QMENTA has agreed to sponsor the event and provide computational resources through their platform.

[ ] Get in touch with QMENTA through their Brain Innovation Hub Slack space, if you think your project would benefit from their support.

ephathaway commented 3 years ago

hi @ohbm/project-monitors: My project is ready!

civier commented 3 years ago

Hello FLOW,

I'm Oren Civier from the AEDAPT project (Australian Electrophysiology Data Analytics PlaTform). Similarly to your platform, the AEDAPT platform will allow users to develop EEG/MEG/neurophysiology analysis pipelines and visualizations with cloud or on-premises server implementations. AEDAPT is based on the lightweight NeuroDesk platform, and thus, can even run on a personal computer. For centralised data management, we will interface with XNAT or BrainLife at first instance.

AEDAPT is not based specifically on Jupyter, but rather provides the user with a full-blown analysis desktop environment (which runs inside a container to provide scalability and reproducibility). Still, tools and libraries are loaded as nested containers, along the same lines of FLOW. For more information, see the Github of the underlying NeuroDesk/VNM platform on which AEDAPT is built: https://github.com/NeuroDesk/vnm/ We also have an ongoing BrainHack project to develop this infrastructure further: https://github.com/ohbm/hackathon2021/issues/2

Given that our projects have some parallels, but also some marked differences, I will be willing to explore opportunities to collaborate, and maybe also to complement each other's tools. More information on the scope of AEDAPT can be found here: https://www.internationalbraininitiative.org/news/australia-brain-data-commons-launches-pilot-project https://ardc.edu.au/news/new-data-projects-will-help-transform-australian-research/ (4th entry)

I tag my colleagues to AEDAPT in case they want to participate in the discussion: @DavidjWhite33 @willw1 @stebo85 @aswinnarayanan @TomEmotion

I'm also attending the BrainHack. My mattermost handle is orenciv

Good luck with your project, Oren

ephathaway commented 3 years ago

Hi @civier, thanks for reaching out!

You are right, our projects do have many similarities. I am interested in learning more about what your desktop analysis environment looks like. I will check out those links you sent. On AEDAPT, can users develop pipelines primarily or solely in Python? Or do you support other languages?

If you're interested in checking out the FLOW interface and playing with the Jupyter Notebooks integration, we can set you up with an account. Just send an email to ian.brown@bel.company and he can get you set up.

stebo85 commented 3 years ago

Dear @ephathaway

AEDAPT is mostly agnostic to languages and we also aim to support matlab based pipelines compiled and executed in the matlab runtime environment :)

FLOW and your eeg-workflows look awesome - would be cool to integrate these and contribute :)

Kind regards Steffen

ephathaway commented 3 years ago

@stebo85, nice! We would also like to add support for matlab pipelines, so that FLOW users can script with EEGLAB. Right now we are limited to Python though.

So, FLOW itself will be a paid product, so that code is closed source, but where we are really hoping to get some collaboration is on the eeg-workflows public repo as well as expanding open-source EEG analysis tools in general. We have been partnering with the MNE-Python folks to expand their support for different file formats and we expect FLOW users to use MNE pretty heavily, as it is very easy to write scripts in the Juypter Notebooks environment.

As far as the scope of Brainhack, we would very much appreciate some feedback from your team on our eeg-workflows repository. I think we have a good start, but we are still at the stage of figuring out what this will become. We would like to make it a useful resource for both FLOW users and non FLOW users. We have posted a few issues to get some interaction going. You may want to start there.

I can see opportunities for collaboration outside of Brainhack as well. I am going to go ahead and give you our CEO's email don.tucker@bel.company. We are working on a number of other products and research projects which you might find interesting, and I am sure he would be happy to give you more details and talk about collaboration opportunities.

ohbm / hackathon2021