ReproNim / neurodocker

Generate custom Docker and Singularity images, and minimize existing containers
https://www.repronim.org/neurodocker/
Apache License 2.0
325 stars 97 forks source link

can't build afni with R #584

Closed andreifoldes closed 9 months ago

andreifoldes commented 9 months ago

Greetings,

I'm trying to run the following code:

singularity run docker://repronim/neurodocker:latest generate singularity \
    --pkg-manager yum \
    --base-image fedora:36 \
    --afni method=binaries version=latest install_r_pkgs=true > /cubric/collab/359_curiosity/tamas/misc/afni-binaries-r.Singularityfile

singularity build --remote /cubric/collab/359_curiosity/tamas/images/afni-binaries-r.sif /cubric/collab/359_curiosity/tamas/misc/afni-binaries-r.Singularityfile

but get the following error

[...]
42 files removed
+ rm -rf '/var/cache/yum/*'
++ find / -name 'libgsl.so.??'
find: File system loop detected; '/tmp/build-temp-1350571086/rootfs' is part of the same file system loop as '/'.
++ printf ''
+ gsl_path=/usr/lib64/libgsl.so.25
+ '[' -n /usr/lib64/libgsl.so.25 ']'
++ dirname /usr/lib64/libgsl.so.25
+ ln -sfv /usr/lib64/libgsl.so.25 /usr/lib64/libgsl.so.0
'/usr/lib64/libgsl.so.0' -> '/usr/lib64/libgsl.so.25'
+ ldconfig
+ mkdir -p /opt/afni-latest
Downloading AFNI ...
+ echo 'Downloading AFNI ...'
+ tar -xz -C /opt/afni-latest --strip-components 1
+ curl -fL https://afni.nimh.nih.gov/pub/dist/tgz/linux_openmp_64.tgz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1092M  100 1092M    0     0  26.8M      0  0:00:40  0:00:40 --:--:-- 29.5M
+ yum install -y -q R-devel
+ yum clean all
42 files removed
+ rm -rf '/var/cache/yum/*'
+ rPkgsInstall -pkgs ALL
/.post.script: line 47: rPkgsInstall: command not found
FATAL:   While performing build: while running engine: exit status 127
FATAL:   While performing build: build image size <= 0
Remi-Gau commented 9 months ago

It seems that the R install of AFNI is not something we test regularly: https://github.com/ReproNim/neurodocker/actions/runs/6520678281/job/17708466328

We should probably fix that.

That being said AFNI is the most "broken" install that neurodocker has at the moment so any help to improve that is most welcome

stebo85 commented 9 months ago

Dear @andreifoldes,

You can have a look at this neurodocker recipe, which works for installing the R-packages in AFNI: https://github.com/NeuroDesk/neurocontainers/blob/master/recipes/afni/build.sh

In particular, you could try fedora 35 as a base image, which I have found to work best for AFNI:

   --base-image fedora:35 \
   --pkg-manager yum \
   --afni version=latest method=binaries install_r_pkgs='true' install_python3='true'

The error you get might also be related to singularity - it could be that building this with docker would actually work. It seems that it can't find the executable rPkgsInstall on the PATH, which could be a path error that somehow works in Docker?

If you just want an AFNI container with all required dependencies and R-packages installed and tested, you can also use the singularity container from the Neurodesk project (https://www.neurodesk.org/docs/getting-started/neurocontainers/singularity/):

curl -X GET https://objectstorage.us-ashburn-1.oraclecloud.com/n/sd63xuke79z3/b/neurodesk/o/afni_23.0.07_20230302.simg -O

I hope that helps Kind regards Steffen

andreifoldes commented 9 months ago

Greetings+

Manually editing the recipe from from rPkgsInstall -pkgs ALL to /opt/afni-latest/rPkgsInstall -pkgs ALL seems to have done the trick!

The error you get might also be related to singularity - it could be that building this with docker would actually work. Unfortunately my cluster only support Singularity :(

If you just want an AFNI container with all required dependencies and R-packages installed and tested, you can also use the singularity container from the Neurodesk project (https://www.neurodesk.org/docs/getting-started/neurocontainers/singularity/)

Amazing! Thx I see that it contains python too!

andreifoldes commented 9 months ago

I stand corrected... while adding full path in the recipe does get the build issue fixed, once actually trying it out on real life data produces:

Loading required package: lme4
Loading required package: Matrix
Error: package or namespace load failed for 'lme4':
 package 'lme4' was installed before R 4.0.0: please re-install it
Failed with error:  'package 'lme4' could not be loaded'

This occurs with the neurodesk simg file specifically!

In terms of neurodocker - can yum install specific version of R? I have a feeling that if only I could get R 3.6 all 3dLMEr would work.

stebo85 commented 9 months ago

Dear @andreifoldes,

neurodocker cannot install a specific version of R through yum - the best way I found to get a specific version of R is to choose a baseimage version that brings this R version. Another option could be to install R manually in a specific version.

So, 3dLMEr only works with R 3.6? Is that correct?

andreifoldes commented 9 months ago

choose a baseimage version that brings this R version

I see! Can you recommend which baseimage would be it for R 3.6, how could I check?

So, 3dLMEr only works with R 3.6? Is that correct?

I haven't checked this in all Linux distributions, but we have Centos7 on our cluster and there I had trouble getting lme4 working with r-base>=4 and had better success with 3.6. 3dLMEr needs lme4, lmerTest, phia and snow to work. In terms of singularity image I would be happy to have any working solution ^^

Another option could be to install R manually in a specific version.

Right, but then I would need to install it every time I call the function right? Or would you turn it into a sandbox?

stebo85 commented 9 months ago

I think we can close this issue here. As discussed in this issue (https://github.com/NeuroDesk/neurocontainers/issues/505), the problems @andreifoldes experienced come from running the singularity container without --cleanenv and R-packages from the host environment interfered with the container environment.