This pull request contains a bunch of improvements.
Critical bugs in PUMI were found and solved
Not explicitly specifying MultiProc could lead to resource problems
If the plugin argument was not explicitly specified, then the CLI argument parser did not pass the nipype runtime arguments to nipype, resulting in the usage of nearly all recourses.
Bug in bbr-argument
Another critical bug was in the parsing of the bbr argument, which lead to the situation, that bbr was never used, even though it should be used per default.
CLI help argument did not work
It was not possible to build our Dockerfile(s)
FSL 6.0.1 installation problems as well as conda related problems lead to the Dockerfile(s) build crashing.
These problems were fixed.
Requirement related problems
A couple of python package requirement related problems were fixed.
The pyproject.toml was missing a couple of requirements as well as the requirements.txt.
Could not install PUMI with Python 3.11
Now, it's possible to install PUMI with Python 3.11
Adapt RPN model for scikit-learn 1.3.2
We used scikit-learn 1.1.2, but this version won't work with upcoming Python versions.
Because of this reason, I upgraded from scikit-learn 1.1.2 to 1.3.2.
Unfortunately, the RPN model did not work with 1.3.2, a slight modification of the model needed to be done.
Some new features were also introduced
Tool versions are now also sinked
It's important to be able to identify which PUMI version was used to create the given derivatives, since pipelines might have changed over time. It's also interesting to know which tool version was used (e.g., FSL version).
Per default, these versions will now be sinked into the file called 'software_versions.txt' (for the currently existing pipelines).
cluster-pipeline-executor.sh
The new script allows you to run PUMI pipelines on the cluster efficiently. It creates one job per subject and is scalable across many cluster nodes, also due to a strategy that reduces the chance of network congestion as much as possible.
Some additional things that were done:
Rework CLI argument parser
Since the CLI argument parser code was not well readable, I reworked the code and added more documentation.
Update Python version in Docker image from 3.8 to 3.11
Update FSL version inside of Docker image from 6.0.1 to 6.0.5
Add missing scripts to pyproject.toml
Issues that were fixed:
close #182
close #181
close #180
close #178
close #177
close #176
close #175
close #170
close #156
This pull request contains a bunch of improvements.
Critical bugs in PUMI were found and solved
Not explicitly specifying MultiProc could lead to resource problems
If the plugin argument was not explicitly specified, then the CLI argument parser did not pass the nipype runtime arguments to nipype, resulting in the usage of nearly all recourses.
Bug in bbr-argument
Another critical bug was in the parsing of the bbr argument, which lead to the situation, that bbr was never used, even though it should be used per default.
CLI help argument did not work
It was not possible to build our Dockerfile(s)
FSL 6.0.1 installation problems as well as conda related problems lead to the Dockerfile(s) build crashing. These problems were fixed.
Requirement related problems
A couple of python package requirement related problems were fixed. The pyproject.toml was missing a couple of requirements as well as the requirements.txt.
Could not install PUMI with Python 3.11
Now, it's possible to install PUMI with Python 3.11
Adapt RPN model for scikit-learn 1.3.2
We used scikit-learn 1.1.2, but this version won't work with upcoming Python versions. Because of this reason, I upgraded from scikit-learn 1.1.2 to 1.3.2. Unfortunately, the RPN model did not work with 1.3.2, a slight modification of the model needed to be done.
Some new features were also introduced
Tool versions are now also sinked
It's important to be able to identify which PUMI version was used to create the given derivatives, since pipelines might have changed over time. It's also interesting to know which tool version was used (e.g., FSL version). Per default, these versions will now be sinked into the file called 'software_versions.txt' (for the currently existing pipelines).
cluster-pipeline-executor.sh
The new script allows you to run PUMI pipelines on the cluster efficiently. It creates one job per subject and is scalable across many cluster nodes, also due to a strategy that reduces the chance of network congestion as much as possible.
Some additional things that were done:
Rework CLI argument parser
Since the CLI argument parser code was not well readable, I reworked the code and added more documentation.
Update Python version in Docker image from 3.8 to 3.11
Update FSL version inside of Docker image from 6.0.1 to 6.0.5
Add missing scripts to pyproject.toml
Issues that were fixed:
close #182 close #181 close #180 close #178 close #177 close #176 close #175 close #170 close #156