Apptainer / Singularity insufficient build encapsulation

[mslw]

I can build the container image with apptainer (everything applies for singularity, too):

❱ apptainer build --fakeroot icf.sif singularity/icf.def
(...)
INFO:    Build complete: icf.sif

When, for some reason, I want to build the image again, it fails:

❱ apptainer build --fakeroot icf2.sif singularity/icf.def
(...)
2024-06-11T17:12:28+0000 [INFO    ] datalad_installer: git-annex is now installed at /tmp/dl-build-5n4aqld0/git-annex.linux/git-annex
+ cut -d : -f 1
+ cut -d = -f 2
+ mv /tmp/dl-build-c26wd0t_/git-annex.linux /tmp/dl-build-5n4aqld0/git-annex.linux /git-annex
mv: target '/git-annex': No such file or directory
FATAL:   While performing build: while running engine: exit status 1

The runs leave behind, in the host system's /tmp these files, produced by datalad-installer:

❱ ls /tmp | grep dl
dl-build-5n4aqld0
dl-build-c26wd0t_
dlinstaller_env.sh
❱ cat /tmp/dlinstaller_env.sh
export PATH=/tmp/dl-build-c26wd0t_/git-annex.linux:"$PATH"
export PATH=/tmp/dl-build-5n4aqld0/git-annex.linux:"$PATH"

Removing the /tmp/dlinstaller_env.sh file allows the new build to proceed:

❱ rm /tmp/dlinstaller_env.sh
❱ apptainer build --fakeroot icf2.sif singularity/icf.def
(...)
INFO:    Build complete: icf2.sif

Crucially, the same would hapen if, e.g. /tmp/dlinstaller_env.sh was already present for unrelated reasons (e,g. created by datalad-installer).

This is probably not very alarming (given that /tmp is usually wiped on reboot) but maybe we can improve on build encapsulation.

Note: datalad-installer appends to the /tmp/dlinstaller_env.sh file; the cut command used in the definition file to get the temporary directory produces two lines of output if the file has two lines.

Reported by Stefanie

[mslw]

Note: datalad-installer appends to the /tmp/dlinstaller_env.sh file; the cut command used in the definition file to get the temporary directory produces two lines of output if the file has two lines.

this should work:

tail -n 1 /tmp/dlinstaller_env.sh | cut -d = -f 2 | cut -d : -f 1

but I am also tempting to do what datalad-installer does:

2024-07-02T15:22:53+0000 [INFO    ] datalad_installer: Installing git-annex via snapshot
2024-07-02T15:22:53+0000 [INFO    ] datalad_installer: Downloading and extracting under /tmp/dl-build-xu2_movo/git-annex.linux
2024-07-02T15:22:53+0000 [INFO    ] datalad_installer: Downloading https://downloads.kitenet.net/git-annex/linux/current/git-annex-standalone-amd64.tar.gz
2024-07-02T15:23:05+0000 [INFO    ] datalad_installer: Running: tar -C /tmp/dl-build-xu2_movo -xzf /tmp/dl-build-xu2_movo/git-annex-standalone-amd64.tar.gz
2024-07-02T15:23:07+0000 [INFO    ] datalad_installer: git-annex is now installed at /tmp/dl-build-xu2_movo/git-annex.linux/git-annex

(plus, probably, setting the env variable explicitly)