Add new supported libraries in Capsul (AFNI and ANTS)

[LStruber]

In order to implement new pipelines, such as MRIQC (https://mriqc.readthedocs.io/en/stable/workflows.html), I would need to use nypipe bricks coming from AFNI and ANTS.

For now, these libraries are not supported by Capsul nor MIA (as opposed to FSL or SPM). What is the scripting effort necessary to support a new library (add it in settings, check if library is configured in init, and run properly the brick) ?

I noticed that, e.g. for fsl, there is a fsl.py script in engine.module, in in_context and in subprocess. What is the purpose of each one ? Besides, fsl is configured/activated in in_context/nypipe.py, this should be done for new nypipe modules also, right ?

In other (simpler) words, I would take advantage to have some advice on how I should proceed to add a new library...

[denisri]

Hi, Yes you're right: we don't have config modules for AFNI and ANTS yet. I'm not used to them and I don't know what is needed to configure them. In the simplest case they just need to be in the PATH to access executable files, but it may be more complex (for instance FSL and Freesurfer require some other environment variables, an init script, etc). Things to do to define a new config module:

• create a module in capsul.engine.module, for instance afni.py. It should add config options for the software (typically path where it is installed), a function activate_configurations which will be called at runtime to apply the configuration options (like add a directory in the PATH env variable, for instance), and an edition_widget() function for the GUI. You can copy what is in the fsl.py module for instance.
• the nipype module should activate the new modules in its activate_configurations() function: just add it in the list in the loop there
• nipype module activate_configurations() uses functions defined in capsul.in_context.nipype, you may have to add functions for AFNI and ANTS there too. I don' remember why the in_context part was separated and if it's needed to keep them separated, maybe @sapetnioc has a better memory...

[LStruber]

Thank, I'll look at it into more details. What about capsul.subprocess part ?

[denisri]

capsul.subprocess is "just" helper functions for runtime commands to call the given software in a subprocess-like API (Popen, call(), check_call(), check_output() etc). They wrap the configuration options and call the needed init scripts in a transparent way. But if the software is "simple enough", it's not needed, and moreover nipype has its own way of calling software, thus will not make use of it.

[LStruber]

I'm able to run AFNI scripts from my host machine, but when I try to run them from bv container, AFNI (which is installed outside the container) throw an error on missing libraries (libxm4).

This is indeed a required library for AFNI which is not installed into the container. Here is the complete list of prerequisites: https://afni.nimh.nih.gov/pub/dist/doc/htmldoc/background_install/install_instructs/steps_linux_ubuntu20.html#slow-setup-install-prerequisite-packages

Can I safely install these libraries into the container ? I think for example to "python-is-python3" that will maps all python calls to python 3, can it be a problem ?

Then, how should we proceed for future users that would potentially use AFNI using bv container ?

[LStruber]

By installing the prerequisites (into my casa-dev singularity sandbox) I'm now able to launch an AFNI brick from inside the bv container (in command line)!

[LStruber]

@denisri I added configuration for AFNI into a new capsul branch (https://github.com/populse/capsul/tree/issue%23205_new_libraries). However, since I cannot test every cases in my configuration and that MIA does not cross every created functions I'm not completely sure that I did everything as it should be done. Would you have time to take a look in the branch to check a bit what I've done ? Note that for now UT fails for a reason I do not understand yet, but I will investigate...

For information, I now have a MIA branch (https://github.com/populse/populse_mia/tree/issue%23205(Capsul)_new_libraries) on my station where I can set the AFNI path into the settings, and run a 3dSkullStrip AFNI brick implemented in MIA processes (local branch only for now)

[denisri]

Hi,

For the prerequisites, there are 2 aspects I think: a. are they big (big enough to make images significantly larger) ? b. how many software will we have to maintain prerequisites support for ?

a. If it is only for libxm4 (I thought Motif was dead for decades but apparently not ;) ) I think we can add it in our main images. python-is-python3 could probably be installed in python3-based distributions. b. We cannot take into account every possible software which needs adding dependencies every day. We should have a "reasonable" basis, but for other software I think users will have to install a read-write distribution and add things by their side. I would like to have a simple modular solution, but singularity, with its read-only images, doesn't allow that. For vbox or docker-based distributions, it could be possible to trigger an install script, but we would have to provide it, and it would not work in all situations...

For your branch for the Afni modules in Capsul, It looks OK, but as I am not using Afni, and I don't really know how we launch it, I don't know if it works. But as long as it doesn't break anything else I guess you can merge it into master. I see you have also written a StudyConfig module: this one was perhaps not necessary, as StudyConfig is obsolete and we are trying to get rid of it, but it's OK.

[denisri]

is libxm4 enough, or should we install all dependencies cited in the page you point ? They include "lots" of things such as firefox, which doesn't seem to be "minimalist" ?

[LStruber]

Obviously Firefox is a good example of dependencies in the list that is not needed for AFNI to run, and there are several others such as gnome-terminal, nautilus etc. However libxm4 isn't enough to run AFNI bricks (I'm sure that libglw1-mesa is also necessary, and certainly some other libraries).

I will check in a fresh casa-distro distribution which libraries it is necessary to add (by installing them one by one) for AFNI to run.

Regarding my branch, thank you for watching. Fyi AFNI only need a path where bin are installed to run. Anyway, as the UT doesn't pass, I do not merge into master for now.

When testing for afni configuration, it fails while creating a new configuration with this error:

OSError: Cannot create configurations for environment "my_machine" because global settings returned 2 instances for module capsul.engine.module.afni

And I don't know yet why there is "2" instances for afni. It sounds like a "default" configuration is present before I create one but it is not the case for fsl or spm, then I have to dig a bit to understand that. If you have a guess, it is welcome... When UT will pass I'll merge the branch into master.

[denisri]

It's OK, the test failed because the initial afni config was not deleted before a new one is added, thus there were 2 of them. I used the same as for fsl. You can see the diffs in the commit above. Thus tests pass now.

[LStruber]

whoops I actually saw that you did this for FSL but I thought it was just to test if the config could be removed ! Thanks

[LStruber]

closed via commits 097e9e6ef7b4ca7d06b5a21671bcab905bc95cf8 and 820fb70c6b341dbbadeabb07a9938c314fe73689