search

ReproNim / neurodocker

Generate custom Docker and Singularity images, and minimize existing containers
https://www.repronim.org/neurodocker/
Apache License 2.0
329 stars 97 forks source link

CAT12 and SPM don't like each other with neurodocker:0.9.4 #510

Open JohannesWiesner opened 1 year ago

JohannesWiesner commented 1 year ago

I would like to add both CAT12 and SPM to our docker image. Installing both independently works fine, but when adding both I get the following error:

Downloading MATLAB Compiler Runtime ...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   169    0   169    0     0   1270      0 --:--:-- --:--:-- --:--:--  1280
100  209M  100  209M    0     0  14.7M      0  0:00:14  0:00:14 --:--:-- 16.7M
Downloading standalone SPM12 ...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   274  100   274    0     0   1683      0 --:--:-- --:--:-- --:--:--  1691
100 99.9M  100 99.9M    0     0  7838k      0  0:00:13  0:00:13 --:--:-- 9032k

------------------------------------------------------------------------
       Segmentation violation detected at Thu Feb  9 14:08:10 2023
------------------------------------------------------------------------

Configuration:
  Crash Decoding      : Disabled - No sandbox or build area path
  Crash Mode          : continue (default)
  Default Encoding    : US-ASCII
  GNU C Library       : 2.24 stable
  Host Name           : 9bef4cbe2bef
  MATLAB Architecture : glnxa64
  MATLAB Root         : /opt/MCR-2017b/v93
  MATLAB Version      : 9.3.0.713579 (R2017b)
  Operating System    : Linux 5.15.0-52-generic #58-Ubuntu SMP Thu Oct 13 08:03:55 UTC 2022 x86_64
  Processor ID        : x86 Family 6 Model 141 Stepping 1, GenuineIntel

Fault Count: 1

Abnormal termination:
Segmentation violation

Register State (from fault):
  RAX = 0000000000000000  RBX = 00007ffd97813330
  RCX = 0000000000000000  RDX = 00000000006030a0
  RSP = 00007ffd978130f0  RBP = 00007ffd97813290
  RSI = 0000000000000001  RDI = 00007ffd97813330

   R8 = 0000000000000000   R9 = 0000000000400a95
  R10 = 000000000000007a  R11 = 00007fcd59e70ad0
  R12 = 00007ffd97813330  R13 = 0000000000000001
  R14 = 0000000000000000  R15 = 0000000000400a95

  RIP = 00007fcd3ea0e5e3  EFL = 0000000000010202

   CS = 0033   FS = 0000   GS = 0000

Stack Trace (from fault):
[  0] 0x00007fcd3ea0e5e3                         bin/glnxa64/libmwmclmcr.so+02127331
[  1] 0x00007fcd3ea08efd                         bin/glnxa64/libmwmclmcr.so+02105085 mclInitializeComponentInstanceNonEmbeddedStandalone+00000061
[  2] 0x00007fcd59e70b1a               runtime/glnxa64/libmwmclmcrrt.so.9.3+00109338 mclInitializeComponentInstanceNonEmbeddedStandalone_proxy+00000074
[  3] 0x0000000000400b46                     /opt/spm12-r7771/spm12_glnxa64+00002886
[  4] 0x0000000000400b64                     /opt/spm12-r7771/spm12_glnxa64+00002916
[  5] 0x0000000000400bb2                     /opt/spm12-r7771/spm12_glnxa64+00002994
[  6] 0x00007fcd52574240                        bin/glnxa64/libmwmclbase.so+00385600
[  7] 0x00007fcd3ddaa735                       bin/glnxa64/libmwMVMLocal.so+00448309 _ZN5boost6detail8function21function_obj_invoker0INS_3_bi6bind_tINS3_11unspecifiedENS_9function1IiRKSt6vectorISbIDsSt11char_traitsIDsESaIDsEESaISB_EEEENS3_5list1INS3_5valueISD_EEEEEEiE6invokeERNS1_15function_bufferE+00000037
[  8] 0x00007fcd52297239                            bin/glnxa64/libmwmcr.so+00586297
[  9] 0x00007fcd52297788                            bin/glnxa64/libmwmcr.so+00587656
[ 10] 0x00007fcd52297a93                            bin/glnxa64/libmwmcr.so+00588435 _Z12mcr_run_mainRKN5boost9function0IiEEbb+00000243
[ 11] 0x00007fcd3dda55a8                       bin/glnxa64/libmwMVMLocal.so+00427432
[ 12] 0x00007fcd51f5038d                            bin/glnxa64/libmwmvm.so+02507661
[ 13] 0x00007fcd525746bd                        bin/glnxa64/libmwmclbase.so+00386749 mclRunMain+00000989
[ 14] 0x0000000000400c59                     /opt/spm12-r7771/spm12_glnxa64+00003161
[ 15] 0x00007fcd595b62e1                    /lib/x86_64-linux-gnu/libc.so.6+00131809 __libc_start_main+00000241
[ 16] 0x00000000004009ba                     /opt/spm12-r7771/spm12_glnxa64+00002490
[ 17] 0x0000000000000000                                   <unknown-module>+00000000

If this problem is reproducible, please submit a Service Request via:
    http://www.mathworks.com/support/contact_us/

A technical support engineer might contact you with further information.

Thank you for your help.** This crash report has been saved to disk as /root/matlab_crash_dump.19798-1 **

MATLAB is exiting because of fatal error
Killed
The command '/bin/sh -c export TMPDIR="$(mktemp -d)"     && apt-get update -qq     && apt-get install -y -q --no-install-recommends            bc            ca-certificates            curl            libncurses5            libxext6            libxmu6            libxpm-dev            libxt6            multiarch-support            unzip     && rm -rf /var/lib/apt/lists/*     && _reproenv_tmppath="$(mktemp -t tmp.XXXXXXXXXX.deb)"     && curl -fsSL --retry 5 -o "${_reproenv_tmppath}" http://mirrors.kernel.org/debian/pool/main/libx/libxp/libxp6_1.0.2-2_amd64.deb     && apt-get install --yes -q "${_reproenv_tmppath}"     && rm "${_reproenv_tmppath}"     && apt-get update -qq     && apt-get install --yes --quiet --fix-missing     && rm -rf /var/lib/apt/lists/*     && echo "Downloading MATLAB Compiler Runtime ..."     && curl -fL -o "$TMPDIR/MCRInstaller.bin" https://dl.dropbox.com/s/zz6me0c3v4yq5fd/MCR_R2010a_glnxa64_installer.bin     && chmod +x "$TMPDIR/MCRInstaller.bin"     && "$TMPDIR/MCRInstaller.bin" -silent -P installLocation="/opt/matlab-compiler-runtime-2010a"     && rm -rf "$TMPDIR"     && unset TMPDIR     && echo "Downloading standalone SPM12 ..."     && curl -fL -o /tmp/spm12.zip https://www.fil.ion.ucl.ac.uk/spm/download/restricted/utopia/previous/spm12_r7771_R2010a.zip     && unzip -q /tmp/spm12.zip -d /tmp     && mkdir -p /opt/spm12-r7771     && mv /tmp/spm12/* /opt/spm12-r7771/     && chmod -R 777 /opt/spm12-r7771     && rm -rf /tmp/spm*     && /opt/spm12-r7771/run_spm12.sh /opt/matlab-compiler-runtime-2010a/v713 quit' returned a non-zero code: 137

This is the script to generate the Dockerfile. Following the suggestions from the docs, I am using the 2017b MCR version for CAT12:

# function to create a dockerfile
generate_docker() {
    docker run -i --rm repronim/neurodocker:0.9.4 generate docker \
        --base-image neurodebian:stretch-non-free \
        --pkg-manager apt \
        --install opts="--quiet" \
            gcc \
            g++ \
            octave \
        --matlabmcr version=2017b \
        --cat12 version=r1933_R2017b \
        --spm12 version=r7771 \
        --copy $conda_yml_file /tmp/ \
        --miniconda \
            version=latest \
            yaml_file=/tmp/$conda_yml_file \
            env_name=csp \
        --user csp \
        --run 'mkdir /home/csp/data && chmod 777 /home/csp/data && chmod a+s /home/csp/data' \
        --run 'mkdir /home/csp/output && chmod 777 /home/csp/output && chmod a+s /home/csp/output' \
        --run 'mkdir /home/csp/code && chmod 777 /home/csp/code && chmod a+s /home/csp/code' \
        --run 'mkdir /home/csp/.jupyter && echo c.NotebookApp.ip = \"0.0.0.0\" > home/csp/.jupyter/jupyter_notebook_config.py' \
        --workdir /home/csp/code \
        --run 'echo source activate csp >> /home/csp/.bashrc'
}

# generate Dockerfile
generate_docker > Dockerfile
}

As described above, deleting either

--matlabmcr version=2017b \
--cat12 version=r1933_R2017b \

or

--spm12 version=r7771 \

will work fine.

stebo85 commented 1 year ago

Dear @JohannesWiesner,

I ran into the same problem when building spm12 and cat12 for the Neurodesk project. The problem is that the neurodocker SPM recipe installs MCR_R2010a (and that's hardcoded) and CAT12 currently requires R2017b (that's not installed via the neurodocker build script).

If you really want both in the same container you could install them manually with matching MCR versions by combining these two build scripts: https://github.com/NeuroDesk/neurocontainers/blob/master/recipes/spm12/build.sh https://github.com/ReproNim/neurodocker/blob/master/neurodocker/templates/cat12.yaml

CAT12, however, only exists for R2017b: http://141.35.69.218/cat12/?C=M;O=D

so you need to change SPM to this as well.

The MCRs are somewhat backward compatible, but ideally, you want to match the versions exactly.

I hope that helps Cheers Steffen

JohannesWiesner commented 1 year ago

Ah, I see. Would it help if both CAT12 and SPM get their own/private MCR (thus increasing the final image size, but possibly making the building process less prone to errors?)

satra commented 1 year ago

that could be one consideration. another is also perhaps adjusting the requirements in the templates to a common mcr would allow it to work together even without an external mcr line. i think this is also true of freesurfer, which has some matlab components. would welcome PRs.

in the longer term we need to essentially do something like conda/mamba's conflict resolution detection across packages.

JohannesWiesner commented 1 year ago

Common MCR sounds like a fancy idea, but then the script would have to be smart enough to detect if SPM/MCR is already present, which came first, and since CAT12 only works with MCR2017b it would have to throw an error if SPM was installed with a non-compatible MCR version. I would rather wait until Christian Gaser also provides more recent MCR versions and pay the price of a bigger image but ensuring a more modular building process for now. Then the installation of SPM and CAT12 could be specified as:

--spm12 version=r7771 \
--cat12 version=r1933_R2017b \

without the user having to specify additionally:

--matlabmcr version=2017b

since this would then happen automatically in the background once --cat12 version=r1933_R2017b is provided.

stebo85 commented 1 year ago

Dear @JohannesWiesner ,

Another problem that arises from installing both SPM12 and cat12 in the same container is: The executable "run_spm12.sh" is identical for SPM and CAT - so it might not be nice to use even if you get it to work with matching MCRs?

We "solved" this problem in the www.Neurodesk.org project by installing SPM12 and CAT12 in separate containers and we then use lmod to handle the paths. Have you thought about doing that or using Neurodesk to achieve what you want? Have look here and see if that works for you: https://play.neurodesk.org/

Cheers Steffen

djarecka commented 1 year ago

@satra - I've opened new issue #513 re. the conflict in conda/mamba

@JohannesWiesner - did you menage to build your container?

Should we close the issue?

JohannesWiesner commented 1 year ago

@djarecka, no I haven't so far. But how is #513 related to this issue?

djarecka commented 1 year ago

513 was mentioned in this conversation

JohannesWiesner commented 1 year ago

@djarecka , sorry I am confused. You linked this issue to your conda/mamba issue, but I don't see how the two are related. This issue deals with conflicts of a parallel installation of CAT12 and SPM12. Mamba/conda have nothing to do with this unless I have overseen something?

djarecka commented 1 year ago

please read: https://github.com/ReproNim/neurodocker/issues/510#issuecomment-1425919655

github-actions[bot] commented 1 year ago

This issue is stale because it has been open for 30 days with no activity.

github-actions[bot] commented 1 year ago

This issue was closed because it has been inactive for 14 days since being marked as stale.

github-actions[bot] commented 12 months ago

This issue is stale because it has been open for 30 days with no activity.