Open tsalo opened 4 years ago
One sticking point for me has been that heudiconv's temporary directories have ended up in weird places, since it uses TMPDIR
to decide where they should go. I got around this by setting TMPDIR
in the cis-bidsify workflow. However, I recently became aware of SINGULARITYENV_*
, which can be set in conjunction with --cleanenv
.
Here's an example use:
$ export SINGULARITYENV_FOO=bar
$ singularity exec --cleanenv alpine_latest.sif env
SHLVL=1
LD_LIBRARY_PATH=/.singularity.d/libs
HOME=/home/vagrant
PS1=Singularity>
TERM=xterm-256color
PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin
FOO=bar
LANG=C
SINGULARITY_APPNAME=
SINGULARITY_CONTAINER=/home/vagrant/alpine_latest.sif
PWD=/home/vagrant
SINGULARITY_NAME=alpine_latest.sif
Note that FOO is set within the container based on SINGULARITYENV_FOO from outside the container, even though we used cleanenv.
Example usage would be
tmpdir=/home/data/nbc/Laird_DIVA/work-dir/${sub}-${ses}/
export SINGULARITYENV_TMPDIR=$tmpdir
singularity exec --cleanenv nipy_heudiconv_latest.sif -d X -s $sub -ss $ses -f X --datalad -b -o X
We could also maybe address #54 with export SINGULARITYENV_PWD
too.
@tsalo, I've kind of flipped my opinion on this. If we remove the heudiconv
processing the package is kind of barebones. I vote that we keep the workflow in it's current state, heudiconv
included.
If our own additions to heudiconv
are small because heudiconv
does almost everything we need, then that's a good thing, in my opinion. A larger tool like heudiconv
should always be our go-to for adding features, and we should only add things to our own workflow when the heudiconv
devs don't want them or the features we need are specific to the CIS or HPC.
Since we figured out that Singularity workaround, there aren't many things in the workflow that are included because they're CIS/HPC-specific. Instead, we can really just focus on the stuff that is outside of heudiconv
's scope.
Now that we have a workaround for operating on datasets in
/home/data
, we don't actually need a lot of the extra work we originally had incis-bidsify
. We've already simplified the workflow, but this has made it clear that we probably don't need to include heudiconv inside our build. Specifically, I'm proposing that we start using a heudiconv Singularity image, followed by a streamlined version ofcis-bidsify
, which would only handle our post-processing steps (i.e., defacing, metadata completion, metadata cleanup, and validation).