add freesurfer 7 and it's minimal form

[satra]

should also test out the minimization enhancement.

[pwighton]

I took a crack at this.

Freesurfer v7.0 release is here

Updating neurodocker was incredibly easy!!

I created a minimal recon-all container following the most excellent the instructions here

Test containers, full and 'recon-only', can be found here:

• pwighton/freesurfer:7.0.0 (6.21GB)
• pwighton/fs-recon-all:7.0.0 (1.33GB)

Some things I'm still trying to figure out:

• What is the best way to get Matlab Compiler Runtime (MCR) 2014b into the 'base' container (see below)?
• How can a script like this be added and invoked by entrypoint, so the FreeSurfer licence file can be passed in as base64 encoded env var?
• The /fsdata could be exclude from the minimized container. Whats the best way to do that?

Building fs-recon-all:7.0.0

Following along here

Get ref data

rsync -aL psydata.ovgu.de::studyforrest/structural/sub-01 /home/paul/cmet/data/studyforrest-data-structural

Generate dockerfile/dockerimage

neurodocker generate docker \
  --base continuumio/miniconda:4.7.12 \
  --pkg-manager apt \
  --freesurfer version=7.0.0 \
    | docker build -t pwighton/freesurfer:7.0.0 -

Start the container

docker run --rm -it --security-opt seccomp:unconfined \
  -v /home/paul/cmet/data/studyforrest-data-structural:/data \
  -v $(pwd)/testdata:/fsdata \
  -e SUBJECTS_DIR='/fsdata' \
  --name fs7 \
  pwighton/freesurfer:7.0.0 \
    /bin/bash

Copy in license.txt

docker cp /home/paul/cmet/license.txt fs7:/opt/freesurfer-7.0.0

Trace commands

cmd1="recon-all -s sub-01 -all -i /data/sub-01/anat/sub-01_T1w.nii.gz -T2 /data/sub-01/anat/sub-01_T2w.nii.gz -T2pial"
cmd2="segmentBS.sh sub-01"
cmd3="recon-all -s sub-01 -hippocampal-subfields-T1T2 /data/sub-01/anat/sub-01_T2w.nii.gz T2 -itkthreads 4"
cmd4="asegstats2table --subjects sub-01 --segno 11 17 18 --meas mean --tablefile aseg.mean-intensity.table"
cmd5="aparcstats2table --hemi lh --subjects sub-01 --tablefile lh.aparc.area.table"

neurodocker reprozip trace \
  --verbosity=debug fs7 \
  "$cmd1" \
  "$cmd4" \
  "$cmd5" \
    | tee neurodocker_reprozip.log

Note: The way brainstem segmentation is invoked ($cmd2) has changed. It has been omitted for now because it requires matlab runtime. Hippo subfeilds also omitted for now.

ERROR: cannot find Matlab 2014b runtime in location:

/opt/freesurfer-7.0.0/MCRv84

It is looking for either:
bin/glnxa64/libmwlaunchermain.so    (Linux 64b) or
bin/maci64/libmwlaunchermain.dylib (Mac 64b)

The hippocampal/amygdala and brainstem modules require the (free) Matlab runtime.
You will need to download the Matlab Compiler Runtime (MCR) for Matlab 2014b.
To do so, please run the following commands (you might need root permissions):

LINUX:

cd /opt/freesurfer-7.0.0
curl "https://surfer.nmr.mgh.harvard.edu/fswiki/MatlabRuntime?action=AttachFile&do=get&target=runtime2014bLinux.tar.gz" -o "runtime2014b.tar.gz"
tar xvf runtime2014b.tar.gz

MAC:

cd /opt/freesurfer-7.0.0
curl "https://surfer.nmr.mgh.harvard.edu/fswiki/MatlabRuntime?action=AttachFile&do=get&target=runtime2014bMAC.tar.gz" -o "runtime2014b.tar.gz"
tar xvf runtime2014b.tar.gz

For further details, please visit:

https://surfer.nmr.mgh.harvard.edu/fswiki/MatlabRuntime

Stop/Delete Container

docker stop fs7

Create minimized Docker image

reprounzip docker setup neurodocker-reprozip.rpz fs7-recon-all

[satra]

@pwighton - thanks for this.

1. regarding MCR can you try this (let us know if it works).

neurodocker generate docker \
  --base continuumio/miniconda:4.7.12 \
  --pkg-manager apt \
  --freesurfer version=7.0.0 \
  --matlabmcr version=2014b install_path=/opt/freesurfer-7.0.0/MCRv84 \
    | docker build -t pwighton/freesurfer:7.0.0 -

2. regarding minimal freesurfer

2a. for current neurodocker release we put the tarball in an external repo. we could use osf.io for that. to do this would require a bit of hacking of the trace files to restrict it to just the /opt/freesurfer* bits. perhaps the same thing can be used to remove fsdata from the container. we would want to add this to support a 7.0.0-min version option to neurodocker.

2b. neurodocker now has some direct functionality for miminizing a container as well (see this section of the README). this is useful when you want to distribute the minimal container.

3. license

you can create an environment variable as below. and then during running the command simply do: docker run -v /path/to/license:/license.txt ...

neurodocker generate docker \
  --base continuumio/miniconda:4.7.12 \
  --pkg-manager apt \
  --freesurfer version=7.0.0 \
  --matlabmcr version=2014b install_path=/opt/freesurfer-7.0.0/MCRv84 \
  --env FS_LICENSE=/license.txt \
    | docker build -t pwighton/freesurfer:7.0.0 -

if you want you can use the bash-command option to stick FS_KEY into the container as well.

[kaczmarj]

thank you @pwighton !

2a. for current neurodocker release we put the tarball in an external repo. we could use osf.io for that. to do this would require a bit of hacking of the trace files to restrict it to just the /opt/freesurfer* bits. perhaps the same thing can be used to remove fsdata from the container. we would want to add this to support a 7.0.0-min version option to neurodocker.

the neurodocker-minify command could be used for this, and then no hacking would have to be done. pseudo-code below

docker run -v /path/to/somewhere:/work --rm -d --name fs IMAGE
neurodocker-minify --container fs -d /opt .... # correct arguments
# delete unused files in /opt after trace is done
docker exec -w /opt tar czf /work/fs7.tar.gz freesurfer-7  # or whatever the directory is called
# now you have the tarball of minimized fs7 in /path/to/somewhere

[pwighton]

Wow, @satra and @kaczmarj!! Neurodocker keeps getting cooler!

Re MCR: Generating the container with

neurodocker generate docker \
  --base continuumio/miniconda:4.7.12 \
  --pkg-manager apt \
  --freesurfer version=7.0.0 \
  --matlabmcr version=2014b install_path=/opt/freesurfer-7.0.0/MCRv84 \
    | docker build -t pwighton/freesurfer:7.0.0 -

Works, but I need to run the following inside the container before running segmentBS.sh sub-01 (even though libmwlaunchermain.so is in LD_LIBRARY_PATH)

mkdir -p /opt/freesurfer-7.0.0/MCRv84/bin/glnxa64/
ln -s /opt/freesurfer-7.0.0/MCRv84/v84/bin/glnxa64/libmwlaunchermain.so /opt/freesurfer-7.0.0/MCRv84/bin/glnxa64/libmwlaunchermain.so

I'll read through the docs tonight and see how much I can clean up.

[satra]

@pwighton - thanks for testing. so one more route :)

neurodocker generate docker \
  --base continuumio/miniconda:4.7.12 \
  --pkg-manager apt \
  --freesurfer version=7.0.0 \
  --matlabmcr version=2014b install_path=/opt/MCRv84 \
  --run "ln -s /opt/MCRv84/v84 /opt/freesurfer-7.0.0/MCRv84" \
    | docker build -t pwighton/freesurfer:7.0.0 -

[pwighton]

Perfect @satra, thanks!

[kaczmarj]

Excellent, and here are updated commands for the minification process. Still untested, but closer to what the commands should be.

docker run -v /path/to/somewhere:/work --rm -d --name fs pwighton/freesurfer:7.0.0

neurodocker-minify --container fs \
  --dirs-to-prune /opt/freesurfer-7.0.0 \
  --commands "$cmd1" "$cmd2" .... etc

# Respond y to delete unused files in /opt/freesurfer-7.0.0 after trace is done

# Rename directory so that in the future when we extract it, it will be `freesurfer`
# like it is with the official freesurfer binaries.
docker exec --workdir /opt fs mv freesurfer-7.0.0 freesurfer

docker exec --workdir /opt \
  tar czf /work/freesurfer-linux-centos6_x86_64-7.0.0-reconall.tar.gz freesurfer
# now you have the tarball of minimized fs7 in /path/to/somewhere

[pwighton]

Thanks @kaczmarj and @satra.

Things seem to be working now. I'm re-running everything to include hippo subfeilds using the following command set:

cmd1="recon-all -s sub-01 -all -i /data/sub-01/anat/sub-01_T1w.nii.gz -T2 /data/sub-01/anat/sub-01_T2w.nii.gz -T2pial"
cmd2="segmentBS.sh sub-01"
cmd3="segmentHA_T2.sh sub-01 /data/sub-01/anat/sub-01_T2w.nii.gz t2 1"
cmd4="asegstats2table --subjects sub-01 --segno 11 17 18 --meas mean --tablefile aseg.mean-intensity.table"
cmd5="aparcstats2table --hemi lh --subjects sub-01 --tablefile lh.aparc.area.table"

[kaczmarj]

fyi a user wanted to use freesurfer 7 in neurodocker, so i merged a pr from him adding the full freesurfer 7 installation in #336. what's left (and much more time-consuming) is adding the minimized freesurfer 7.

[satra]

@kaczmarj - we should add the instructions above to add in support for MCR, otherwise people will not be able to use the full capability of even recon-all - we should have done this even for 6. perhaps add a new flag --add_mcr to the freesurfer install so it does what those additional install steps do.

[kaczmarj]

+1 @satra , i added an issue to track adding --add-mcr to freesurfer

i also remove freesurfer 7.0.0 from neurodocker and replace it with 7.1.0 @pwighton - can you please redo anything you did with the updated version? 7.0.0 had a bug in it, and the freesurfer team recommends not using it. https://surfer.nmr.mgh.harvard.edu/fswiki/ReleaseNotes

[pwighton]

Will do @kaczmarj. Should be able to report back tomorrow.

[ghost]

Hi all,

I tried to follow instructions for freesurfer 7.1.0 with

neurodocker generate docker --base continuumio/miniconda:4.7.12 \
--pkg-manager apt \
--freesurfer version=7.1.0 \
--matlabmcr version=2014b install_path=/opt/MCRv84 \
--run "ln -s /opt/MCRv84/v84 /opt/freesurfer-7.1.0/MCRv84" \
  | docker build -t freesurfer -

but when trying to execute some scripts in the docker i got the following errors:

command nu_correct gave me:

bash: nu_correct: command not found

command mri_nu_correct.mni gave me:

I/O warning : failed to load external entity "/opt/freesurfer-7.1.0/docs/xml/mri_nu_correct.mni.help.xml" Document not parsed successfully.

Doing the same with the older freesurfer versions 6.x, however, worked.

Any idea what cause this bug and how to fix it?

[pwighton]

@mchlrnwld, my container pwighton/freesurfer:7.1.0 seems to build the same way as yours:

neurodocker generate docker \
  --base continuumio/miniconda:4.7.12 \
  --pkg-manager apt \
  --freesurfer version=7.1.0 \
  --matlabmcr version=2014b install_path=/opt/MCRv84 \
  --run "ln -s /opt/MCRv84/v84 /opt/freesurfer-7.1.0/MCRv84" \
    | docker build -t pwighton/freesurfer:7.1.0 -

And nu_correct seems fine

docker run --rm -it \
  -v /tmp/fs-subs:/fsdata \
  -v ${HOME}/license.txt:/license.txt \
  -e FS_LICENSE='/license.txt' \
  -e SUBJECTS_DIR='/fsdata' \
  pwighton/freesurfer:7.1.0 \
    nu_correct

I can confirm that mri_nu_correct.mni errors out when trying to print the help screen

This command

docker run --rm -it \
  -v /home/paul/lcn/data/fs-course-20200310/subjects:/fsdata \
  -v ${HOME}/license.txt:/license.txt \
  -e FS_LICENSE='/license.txt' \
  -e SUBJECTS_DIR='/fsdata' \
  --name fs7 \
  pwighton/freesurfer:7.1.0 \
    mri_nu_correct.mni

and this

docker run --rm -it \
  -v /home/paul/lcn/data/fs-course-20200310/subjects:/fsdata \
  -v ${HOME}/license.txt:/license.txt \
  -e FS_LICENSE='/license.txt' \
  -e SUBJECTS_DIR='/fsdata' \
  --name fs7 \
  pwighton/freesurfer:7.1.0 \
    mri_nu_correct.mni --help

gives:

I/O warning : failed to load external entity "/opt/freesurfer-7.1.0/docs/xml/mri_nu_correct.mni.help.xml"
Document not parsed successfully.

But if it is invoked without having to being the help text, it works

mri_nu_correct.mni --i orig.mgz --o ~/delete_me_nu.mgz --n 2 --ants-n4

[pwighton]

@kaczmarj I've build some new FS7.1 containers. Details here.

It's interesting that there's no longer a big size difference between the 7.0.0-min and 7.0.0-min-plus containers. Next iteration, I might just put all the use cases into 7.0.0-min

[satra]

@pwighton - the reason the size may not be very different is that we may want to prune all of /opt, which would remove the mcr for the min version.

[pwighton]

Yeah @satra, I was expecting a decent size difference because I was expecting the MCR to get pruned from min but not min-plus. This is what seemed to happen for FS 7.0.0, but not 7.1.0. Not sure if recon-all uses MCR if it's found or it is used now in 7.1.0, or something else?

[satra]

@pwighton - just checking where things are with this.

[pwighton]

Hey @satra, my latest notes are here. I haven't revisited this in a while.

I have been meaning to try minifying with --dirs-to-prune /opt instead --dirs-to-prune /opt/freesurfer-7.1.0 to see what the container sizes for 7.1.0-min and 7.1.0-min-plus come out to. I'll start those jobs in a few minutes.

I've also been meaning to collect more use cases, especially longitudinal (which I think could go in 7.1.0-min without increasing the size much). Do folks know of publicly available longitudinal sample data? I suppose I could just copy the studyforest sample and create two identical timepoints.

[satra]

thanks @pwighton

re: longitudinal - there are several - the my connectome dataset from russ, the HBN-Serial Scanning Initiative, the midnight scan club.

[pwighton]

Hey @satra, I've been working on a minimal container that includes longitudinal functionality, and I ran into an issue with fs_time that LCN seems to have already addressed in their v7.1.1 release.

I'll try again building a minimal container again, in the meantime I've put in an MR to support 7.1.1 ( #363 )

[pwighton]

Hey @satra, sorry for the lag on this. I've run into more issues with fs_time, but that can be solved by setting the env var FS_TIME_ALLOW=0. I'm trying to generate a minimal container now using the following commands:

cmd1="recon-all -s sub-01 -all -i /data/sub-01/anat/sub-01_T1w.nii.gz -T2 /data/sub-01/anat/sub-01_T2w.nii.gz -T2pial"
cmd2="recon-all -s sub-01b -all -i /data/sub-01/anat/sub-01_T1w.nii.gz -T2 /data/sub-01/anat/sub-01_T2w.nii.gz -T2pial"
cmd3="asegstats2table --subjects sub-01 --segno 11 17 18 --meas mean --tablefile aseg.mean-intensity.table"
cmd4="aparcstats2table --hemi lh --subjects sub-01 --tablefile lh.aparc.area.table"
cmd5="recon-all -base long-base-sub01 -tp sub-01 -tp sub-01b -all"
cmd6="recon-all -long sub-01 long-base-sub01 -all"

I'll report back when it finished

[ghost]

Hi all,

currently still having issues with fs_time, when running the longitudinal freesurfer 7.1.1 pipeline. Did you manage to solve this issue, @pwighton ?

[pwighton]

Hi @mchlrnwld , yes my runs failed as well and then this fell through the cracks. I'll try to find some time to investigate this week and file an issue on the freesurfer repo if needed.

[pwighton]

Hi @mchlrnwld , I checked some old logs, and things seemed to work when the env var FS_TIME_ALLOW is set to 0. I'm re-running things now to confirm.

[pwighton]

Hi @mchlrnwld I just finished running a 'cross', 'base' and 'long' parts of the longitudinal stream in pwighton/freesurfer:7.1.1 using -e FS_TIME_ALLOW=0 without error.

There's some notes here. The longitudinal-related recon commands in those notes (cmd2, cmd3, cmd6 and cmd7) all completed without error.

[pwighton]

There's now a minimized container at pwighton/freesurfer:7.1.1-min created using neurodocker-minify. With support for T1, T2 and longitudinal recons.

[satra]

closed via #370