TrainTrack: Development & distribution of Python scripts using MRtrix3

[Lestropie]

Development & distribution of Python scripts using MRtrix3

Project Description

MRtrix3 provides a set of tools to perform various types of diffusion MRI analyses, from various forms of tractography through to next-generation group-level analyses.

The majority of tools provided within MRtrix3 are built using C++, and hence those underlying APIs are only accessible to researchers with the requisite skills in that language.

More recently however we have incorporated a relatively simple Python API, which is intended for the automation of higher-level image processing tasks that can be achieved using a combination of existing lower-level commands (whether from MRtrix3 or other softwares). Many frequently-used commands provided with MRtrix3 already make use of this API.

It is additionally possible for stand-alone processing scripts to make use of this API, which then inherit the various benefits provided by the API:

• Integrated command-line parsing capability, with an interface identical to MRtrix3 commands;

• Command-line terminal output that is consistent with other MRtrix3 Python scripts, with multiple available levels of terminal verbosity;

• Self-generation of inline paginated help page, as well as Markdown and ReStructured Text documentation;

• Integrated management of scratch directory for intermediate data processing;

• Compatibility with both Python2 and Python3;

• Various convenience functions that have been accumulated over time due to their utility in tasks regularly encountered in the development of such processing scripts; e.g. wrapping functionalities of other software packages, robust parsing of user inputs, provenance management.

Note: This library does not involve the direct manipulation of image data within Python itself; it is purely dedicated to the automation of processing tasks that can be built from a sequence of existing commands.

If there were sufficient interest, I could perform an ad hoc session demonstrating the basic usage of this API, as well as provide support to anybody intending to develop tools using this API during the hackathon.

Skills required to participate

Some requisite experience with Python is necessary; an attendee without such would likely be unable to recognise the distinction between general Python capabilities and the capabilities of this specific API. Beyond that, some familiarity with MRtrix3 would be highly recommended, as knowledge of the appropriate underlying commands for basic image manipulation operations means that time can be focused on the development of higher-level functionalities.

Integration

Processing pipeline projects that are implemented in "raw" Python (i.e. without use of an established API) will tend to run into the very same implementation hurdles that justified the development of the MRtrix3 Python API. By providing a "stepping stone" to the use of this particular API, this TrainTrack may help to fast-track new projects, by avoiding the overhead of these myriad generic scripting challenges, and enabling more rapid commencement of work on the actual novel aspects of any particular project. Scripts developed against this API may later be distributed individually and executed by anyone with a valid MRtrix3 installation, or, if sufficiently novel / relevant / useful, could be integrated into the MRtrix3 package itself.

Preparation material

• The preprint of the MRtrix3 manuscript provides simple example commands in both C++ and Python (see Appendix B).

• The code for those Python scripts provided with MRtrix3 is open-source, and can give some indication of how the API is used. (Note: this hyperlink directs to development branch code, as the Python API will soon be undergoing changes as part of the upcoming "3.0_RC4" tag)

• My BIDS App "MRtrix3_connectome" demonstrates how a relatively large and complex processing pipeline can be fully automated and provided to the research public using this API. (Note: the current version of this App is built against the Python API in MRtrix3 version 3.0_RC3, which is the current public release; this will hopefully be updated to reflect the upcoming API changes prior to the hackathon)

Link to your GitHub repo

• MRtrix3 Python API files (Note: Hyperlink is for development branch, where the most recent API updates currently reside in preparation for tag update 3.0_RC4)

• MRtrix3_connectome BIDS App

Also: Online documentation for those Python scripts currently provided as part of MRtrix3. This documentation is self-generated from the source code, which is one of the benefits of use of this API.

Communication

• MRtrix3 community forum, for general MRtrix3 information and discussion

• My profile on the MRtrix3 community forum; I can be contacted there directly for questions that are specific to the Hackathon and may not be relevant to the MRtrix3 community more generally.

[dnkennedy]

Hi. Is this still a TrainTrack offering that you're willing to offer at the OHBM Hackathon? If so, I'm trying to put offerings into timeslots.

[Lestropie]

Hi Dave,

I've registered for the Hackathon, and am already in Rome, so would still be willing to do an off-the-cuff demonstration. I can't say I've received a huge amount of interest on this TrainTrack specifically, but I don't know what level of pre-event feedback to expect for such things (long-time developer, first-time Hackathon-er).

Of course I'll be loitering around regardless if anyone is either interested in the use of this API, or otherwise wants to grab some of my time during the event.

Rob

[Lestropie]

For anyone interested: I will be explaining the basics of the MRtrix3 Python API for the sake of a late-starting project, and others would be quite welcome to come along and listen in. Today from 1PM on the top floor of the Mercato Centrale.