Inria-Empenn / narps_open_pipelines

A codebase reproducing the 70 pipelines of the NARPS study (Botvinik-Nezer et al., 2020) shared as an open resource for the community.
MIT License
9 stars 22 forks source link

[BUG] Neurodocker failing #209

Open Tritbool opened 1 week ago

Tritbool commented 1 week ago

Describe the bug

Tried to rebuild a docker image from the neurodocker configuration described in environment.md. The generation of Dockerfile is OK with this older version of neurodocker (1.0.0 is suggested at the end of the generation).

However the building of the Docker image simply fails as some packages / versions seems to be now unreachable.

sudo docker build --tag narps_test - < Dockerfile
[+] Building 2.4s (5/17)                                                                                                                                                                                                     docker:default
 => [internal] load build definition from Dockerfile                                                                                                                                                                                   0.0s
 => => transferring dockerfile: 10.73kB                                                                                                                                                                                                0.0s
 => [internal] load metadata for docker.io/library/neurodebian:stretch-non-free                                                                                                                                                        0.7s
 => [internal] load .dockerignore                                                                                                                                                                                                      0.0s
 => => transferring context: 2B                                                                                                                                                                                                        0.0s
 => CACHED [ 1/14] FROM docker.io/library/neurodebian:stretch-non-free@sha256:2a036ad78514c560b4fa3c1f3fca02f6398a4410a764cb8ee8f267c3e841869a                                                                                         0.0s
 => ERROR [ 2/14] RUN export ND_ENTRYPOINT="/neurodocker/startup.sh"     && apt-get update -qq     && apt-get install -y -q --no-install-recommends            apt-utils            bzip2            ca-certificates            curl   1.7s
------                                                                                                                                                                                                                                      
 > [ 2/14] RUN export ND_ENTRYPOINT="/neurodocker/startup.sh"     && apt-get update -qq     && apt-get install -y -q --no-install-recommends            apt-utils            bzip2            ca-certificates            curl            locales            unzip     && apt-get clean     && rm -rf /var/lib/apt/lists/*     && sed -i -e 's/# en_US.UTF-8 UTF-8/en_US.UTF-8 UTF-8/' /etc/locale.gen     && dpkg-reconfigure --frontend=noninteractive locales     && update-locale LANG="en_US.UTF-8"     && chmod 777 /opt && chmod a+s /opt     && mkdir -p /neurodocker     && if [ ! -f "/neurodocker/startup.sh" ]; then          echo '#!/usr/bin/env bash' >> "/neurodocker/startup.sh"     &&   echo 'set -e' >> "/neurodocker/startup.sh"     &&   echo 'export USER="${USER:=`whoami`}"' >> "/neurodocker/startup.sh"     &&   echo 'if [ -n "$1" ]; then "$@"; else /usr/bin/env bash; fi' >> "/neurodocker/startup.sh";     fi     && chmod -R 777 /neurodocker && chmod a+s /neurodocker:                                                                                                                                                                                                                    
1.589 W: The repository 'http://security.debian.org/debian-security stretch/updates Release' does not have a Release file.
1.589 W: The repository 'http://deb.debian.org/debian stretch Release' does not have a Release file.
1.589 W: The repository 'http://deb.debian.org/debian stretch-updates Release' does not have a Release file.
1.589 E: Failed to fetch http://security.debian.org/debian-security/dists/stretch/updates/main/binary-amd64/Packages  404  Not Found [IP: 151.101.130.132 80]
1.589 E: Failed to fetch http://deb.debian.org/debian/dists/stretch/main/binary-amd64/Packages  404  Not Found
1.589 E: Failed to fetch http://deb.debian.org/debian/dists/stretch-updates/non-free/binary-amd64/Packages  404  Not Found
1.589 E: Some index files failed to download. They have been ignored, or old ones used instead.

I also tried using standolone Neurodocker and various neurodebian distros (bionic, bullseye,...). It appears that I had issues with packages or software downloading every time (at various steps in the process) Maybe Neurodocker is problematic for custom builds at the moment.

Reproduction

HW. ENV.

SW. ENV.

STEPS

sudo docker run --rm repronim/neurodocker:0.7.0 generate docker \
           --base neurodebian:stretch-non-free --pkg-manager apt \
           --install git \
           --fsl version=6.0.3 \
           --afni version=latest method=binaries install_r=true install_r_pkgs=true install_python2=true install_python3=true \
           --spm12 version=r7771 method=binaries \
           --user=neuro \
           --workdir /home \
           --miniconda create_env=neuro \
                       conda_install="python=3.8 traits jupyter nilearn graphviz nipype scikit-image" \
                       pip_install="matplotlib" \
                       activate=True \
           --env LD_LIBRARY_PATH="/opt/miniconda-latest/envs/neuro:$LD_LIBRARY_PATH" \
           --run-bash "source activate neuro" \
           --user=root \
           --run 'chmod 777 -Rf /home' \
           --run 'chown -R neuro /home' \
           --user=neuro \
           --run 'mkdir -p ~/.jupyter && echo c.NotebookApp.ip = \"0.0.0.0\" > ~/.jupyter/jupyter_notebook_config.py' > Dockerfile

Then

sudo docker build --tag narps_test - < Dockerfile

Expected behavior

  1. Run neurodocker following the descriptions in environment.md to generate a Dockerfile
  2. Build image using generated Dockerfile
  3. Obtain a docker image

NARPS Open Pipeline version

latest

Operating System

Ubuntu KDE 22.04

Softwares versions

Docker version 26.1.4, build 5650f9b

Screenshots

No response

Remi-Gau commented 1 week ago

I would recommend using the image provided by the project: elodiegermani/open_pipeline

I just checked the status dashboard of the installs with neurodocker (images take a while to load so may need a refresh or 2): https://github.com/ReproNim/neurodocker/blob/test_docker_build/docs/README.md

it's not looking good at the moment.

can only install SPM on centos and AFNI only on fedora so I don't think this recipe will ever build at the moment.

Tritbool commented 6 days ago

This is a solution when one just needs to run pipelines in a stable environment. In my case I want to create different environment with as many different versions of e.g. SPM and MCR to test software versioning impact on pipelines outputs. So I will have to build dockerfiles myself, or go on another solution such as nix.