ReproNim / neurodocker

Generate custom Docker and Singularity images, and minimize existing containers
https://www.repronim.org/neurodocker/
Apache License 2.0
325 stars 97 forks source link

FIX add default header and entrypoint to docker and singularity files #623

Closed mvdoc closed 3 weeks ago

mvdoc commented 1 month ago

Thanks to the archeological work by @yarikoptic in #620 it was relatively easy to fix. The _header.yaml template was not added anywhere. BTW, I had to rename the template from _header to _default because _header is in conflict with SingularityRenderer._header. Some examples before/after below:

Closes #620

BEFORE $ neurodocker generate docker --base-image ubuntu:20.04 --pkg-manager apt ``` # Generated by Neurodocker and Reproenv. FROM ubuntu:20.04 # Save specification to JSON. RUN printf '{ \ "pkg_manager": "apt", \ "existing_users": [ \ "root" \ ], \ "instructions": [ \ { \ "name": "from_", \ "kwds": { \ "base_image": "ubuntu:20.04" \ } \ } \ ] \ }' > /.reproenv.json # End saving to specification to JSON. ```
AFTER $ neurodocker generate docker --base-image ubuntu:20.04 --pkg-manager apt ``` # Generated by Neurodocker and Reproenv. FROM ubuntu:20.04 ENV LANG="en_US.UTF-8" \ LC_ALL="en_US.UTF-8" \ ND_ENTRYPOINT="/neurodocker/startup.sh" RUN export ND_ENTRYPOINT="/neurodocker/startup.sh" \ && apt-get update -qq \ && apt-get install -y -q --no-install-recommends \ apt-utils \ bzip2 \ ca-certificates \ curl \ locales \ unzip \ && rm -rf /var/lib/apt/lists/* \ && sed -i -e 's/# en_US.UTF-8 UTF-8/en_US.UTF-8 UTF-8/' /etc/locale.gen \ && dpkg-reconfigure --frontend=noninteractive locales \ && update-locale LANG="en_US.UTF-8" \ && chmod 777 /opt && chmod a+s /opt \ && mkdir -p /neurodocker \ && if [ ! -f "$ND_ENTRYPOINT" ]; then \ echo '#!/usr/bin/env bash' >> "$ND_ENTRYPOINT" \ && echo 'set -e' >> "$ND_ENTRYPOINT" \ && echo 'export USER="${USER:=`whoami`}"' >> "$ND_ENTRYPOINT" \ && echo 'if [ -n "$1" ]; then "$@"; else /usr/bin/env bash; fi' >> "$ND_ENTRYPOINT"; \ fi \ && chmod -R 777 /neurodocker && chmod a+s /neurodocker ENTRYPOINT ["/neurodocker/startup.sh"] # Save specification to JSON. RUN printf '{ \ "pkg_manager": "apt", \ "existing_users": [ \ "root" \ ], \ "instructions": [ \ { \ "name": "from_", \ "kwds": { \ "base_image": "ubuntu:20.04" \ } \ }, \ { \ "name": "env", \ "kwds": { \ "LANG": "en_US.UTF-8", \ "LC_ALL": "en_US.UTF-8", \ "ND_ENTRYPOINT": "/neurodocker/startup.sh" \ } \ }, \ { \ "name": "run", \ "kwds": { \ "command": "export ND_ENTRYPOINT=\\"/neurodocker/startup.sh\\"\\napt-get update -qq\\napt-get install -y -q --no-install-recommends \\\\\\n apt-utils \\\\\\n bzip2 \\\\\\n ca-certificates \\\\\\n curl \\\\\\n locales \\\\\\n unzip\\nrm -rf /var/lib/apt/lists/*\\nsed -i -e '"'"'s/# en_US.UTF-8 UTF-8/en_US.UTF-8 UTF-8/'"'"' /etc/locale.gen\\ndpkg-reconfigure --frontend=noninteractive locales\\nupdate-locale LANG=\\"en_US.UTF-8\\"\\nchmod 777 /opt && chmod a+s /opt\\nmkdir -p /neurodocker\\nif [ ! -f \\"$ND_ENTRYPOINT\\" ]; then\\n echo '"'"'#!/usr/bin/env bash'"'"' >> \\"$ND_ENTRYPOINT\\"\\n echo '"'"'set -e'"'"' >> \\"$ND_ENTRYPOINT\\"\\n echo '"'"'export USER=\\"${USER:=`whoami`}\\"'"'"' >> \\"$ND_ENTRYPOINT\\"\\n echo '"'"'if [ -n \\"$1\\" ]; then \\"$@\\"; else /usr/bin/env bash; fi'"'"' >> \\"$ND_ENTRYPOINT\\";\\nfi\\nchmod -R 777 /neurodocker && chmod a+s /neurodocker" \ } \ }, \ { \ "name": "entrypoint", \ "kwds": { \ "args": [ \ "/neurodocker/startup.sh" \ ] \ } \ } \ ] \ }' > /.reproenv.json # End saving to specification to JSON. ```
BEFORE $ neurodocker generate singularity --base-image ubuntu:20.04 --pkg-manager apt ``` # Generated by Neurodocker and Reproenv. Bootstrap: docker From: ubuntu:20.04 %post # Save specification to JSON. printf '{ \ "pkg_manager": "apt", \ "existing_users": [ \ "root" \ ], \ "instructions": [ \ { \ "name": "from_", \ "kwds": { \ "base_image": "ubuntu:20.04" \ } \ } \ ] \ }' > /.reproenv.json # End saving to specification to JSON. ```
AFTER $ neurodocker generate singularity --base-image ubuntu:20.04 --pkg-manager apt ``` $ neurodocker generate singularity --base-image ubuntu:20.04 --pkg-manager apt # Generated by Neurodocker and Reproenv. Bootstrap: docker From: ubuntu:20.04 %environment export LANG="en_US.UTF-8" export LC_ALL="en_US.UTF-8" export ND_ENTRYPOINT="/neurodocker/startup.sh" %post export ND_ENTRYPOINT="/neurodocker/startup.sh" apt-get update -qq apt-get install -y -q --no-install-recommends \ apt-utils \ bzip2 \ ca-certificates \ curl \ locales \ unzip rm -rf /var/lib/apt/lists/* sed -i -e 's/# en_US.UTF-8 UTF-8/en_US.UTF-8 UTF-8/' /etc/locale.gen dpkg-reconfigure --frontend=noninteractive locales update-locale LANG="en_US.UTF-8" chmod 777 /opt && chmod a+s /opt mkdir -p /neurodocker if [ ! -f "$ND_ENTRYPOINT" ]; then echo '#!/usr/bin/env bash' >> "$ND_ENTRYPOINT" echo 'set -e' >> "$ND_ENTRYPOINT" echo 'export USER="${USER:=`whoami`}"' >> "$ND_ENTRYPOINT" echo 'if [ -n "$1" ]; then "$@"; else /usr/bin/env bash; fi' >> "$ND_ENTRYPOINT"; fi chmod -R 777 /neurodocker && chmod a+s /neurodocker # Save specification to JSON. printf '{ \ "pkg_manager": "apt", \ "existing_users": [ \ "root" \ ], \ "instructions": [ \ { \ "name": "from_", \ "kwds": { \ "base_image": "ubuntu:20.04" \ } \ }, \ { \ "name": "env", \ "kwds": { \ "LANG": "en_US.UTF-8", \ "LC_ALL": "en_US.UTF-8", \ "ND_ENTRYPOINT": "/neurodocker/startup.sh" \ } \ }, \ { \ "name": "run", \ "kwds": { \ "command": "export ND_ENTRYPOINT=\\"/neurodocker/startup.sh\\"\\napt-get update -qq\\napt-get install -y -q --no-install-recommends \\\\\\n apt-utils \\\\\\n bzip2 \\\\\\n ca-certificates \\\\\\n curl \\\\\\n locales \\\\\\n unzip\\nrm -rf /var/lib/apt/lists/*\\nsed -i -e '"'"'s/# en_US.UTF-8 UTF-8/en_US.UTF-8 UTF-8/'"'"' /etc/locale.gen\\ndpkg-reconfigure --frontend=noninteractive locales\\nupdate-locale LANG=\\"en_US.UTF-8\\"\\nchmod 777 /opt && chmod a+s /opt\\nmkdir -p /neurodocker\\nif [ ! -f \\"$ND_ENTRYPOINT\\" ]; then\\n echo '"'"'#!/usr/bin/env bash'"'"' >> \\"$ND_ENTRYPOINT\\"\\n echo '"'"'set -e'"'"' >> \\"$ND_ENTRYPOINT\\"\\n echo '"'"'export USER=\\"${USER:=`whoami`}\\"'"'"' >> \\"$ND_ENTRYPOINT\\"\\n echo '"'"'if [ -n \\"$1\\" ]; then \\"$@\\"; else /usr/bin/env bash; fi'"'"' >> \\"$ND_ENTRYPOINT\\";\\nfi\\nchmod -R 777 /neurodocker && chmod a+s /neurodocker" \ } \ }, \ { \ "name": "entrypoint", \ "kwds": { \ "args": [ \ "/neurodocker/startup.sh" \ ] \ } \ } \ ] \ }' > /.reproenv.json # End saving to specification to JSON. %runscript /neurodocker/startup.sh ```
codecov[bot] commented 1 month ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 88.06%. Comparing base (b7ddf6f) to head (41a3eef). Report is 22 commits behind head on master.

:exclamation: Current head 41a3eef differs from pull request most recent head 15c5300

Please upload reports for the commit 15c5300 to get more accurate results.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #623 +/- ## ========================================== + Coverage 88.00% 88.06% +0.05% ========================================== Files 11 11 Lines 1034 1039 +5 ========================================== + Hits 910 915 +5 Misses 124 124 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

mvdoc commented 3 weeks ago

@kaczmarj @yarikoptic should we merge?

yarikoptic commented 3 weeks ago

would it be too much to ask to add/extend some test to ensure we do not loose this again going forward?

mvdoc commented 3 weeks ago

@yarikoptic I added a test but now a different test is failing for unrelated reasons (#628)

yarikoptic commented 3 weeks ago

indeed unrelated, updated there, let's proceed here!

yarikoptic commented 3 weeks ago

we should release some time soon!