Automatise your processing pipelines with nipype / pydra

davidmeunier79 commented 1 year ago

Title

Leaders

David Meunier : Twitter: https://twitter.com/DavidM0579 Mattermost: @david.meunier https://mattermost.brainhack.org/brainhack

Collaborators

To fill at the end

Brainhack Global 2022 Event

Brainhack Marseille

Project Description

Neuroimaging and electrophysiology processing requires many steps, calling different softwares, possibly in different languages (typically, matlab batches or shell scripts).

Nipype has provided an integrative solution, with a sufficient level of complexity to cover most of the needs for writting pipelines in neuroimaging. It is based on the notion workflows, being an orderd succession of nodes, linking inputs and outputs. Nodes can be user-written function (in python), interfaces with existing softwares (e.g. FSL, AFNI or SPM), or even other user-defined sub-workflows.

Nipype is at the base of many widely used docker images, such as fmriprep and qsiprep. And has been extendend for other applications, such as EEG/MEG processing (ephypype), graph analysis in functional connectivity (graphpype) or non-human primate anatomical MRI segmentation (macapype).

Nipype has now achieved a degree of maturity to have become predominant in the community. But some of the limitations still prevails. It has decided in the last years to rewrite the core engine of nipype, to incorporate new functionnalities, such as runnnig containers as one node. The new implementation will be called pydra, and also still in its infancy, we expect it to become a major standard in the community.

Link to project repository/sources

Nipype: https://github.com/nipy/nipype

Pydra: https://github.com/nipype/pydra

Goals for Brainhack Global

In this project, we propose :

to give an overview of how nipype works
to advise you if it is useful for your typical processing,
to help writting specific nodes or workflows dedicated to your processing.

For advaced users, We also propose:

to explain the advances of pydra compaired to nipype
to write some tools existing in nipype in pydra.

Good first issues

write an interface of your own python function
build your own very simple processing pipelines with 2 nodes (possibly with your onw function) and one connection using pyBIDS inputs
convert an existing pipeline from nipype to pydra

Communication channels

https://mattermost.brainhack.org/brainhack/channels/bhg22-marseille-auto-nipype-pydra

Skills

Neuroimaging/electrophysiology processing: 100% Shell Script / Matlab Batch: 75% Python: 50% Nipype: 25%

Onboarding documentation

https://macatools.github.io/macapype/contribute.html

What will participants learn?

Writing easily modifiable and reusable pipeline; Contributing to reproducible science

Data to use

None

Number of collaborators

1

Credit to collaborators

Starting a new pydra-based project will be rewarded as a main contributor to the project, and possibly as an author on a subsequent prospective (methological) article. Working on your pipeline will be rewarded as your own github project.

Image

graph

Type

pipeline_development

Development status

0_concept_no_content

Topic

reproducible_scientific_methods

Tools

Nipype

Programming language

Python

Modalities

fMRI

Git skills

2_branches_PRs

Anything else?

Testing is project template is directly usable on the website BrainHack Marseille 2022, Subject to modification.

Things to do after the project is submitted and ready to review.

[x] Add a comment below the main post of your issue saying: Hi @brainhackorg/project-monitors my project is ready!
[x] Twitter-sized summary of your project pitch.

davidmeunier79 commented 1 year ago

@Brainhack-Marseille

davidmeunier79 commented 1 year ago

Hi @brainhackorg/project-monitors my project is ready!

We need to test if everything is matching for our own Brainhack-Marseille event, but we need to see how it works after your approval! Thanks

Remi-Gau commented 1 year ago

Things look good to me but I am unsure whether this is a project within macapype or something more generic where the aim is to contribute back to pydra or something else. Or maybe I misread it.

davidmeunier79 commented 1 year ago

@Remi-Gau Indeed it is more than macapype or neuropycon; if people come with shell or matlab scripts the idea would be to help them to build nipype pipeline from it; if people are skilled in nipype already, there is room for working on nipype-> pydra conversion. Does that make sense to you?

Remi-Gau commented 1 year ago

yup makes sense to me!

Remi-Gau commented 1 year ago

@davidmeunier79

Your project is on the website: https://brainhack.org/global2022/projects/

brainhackorg / global2022