2.4-dist on alpine linux failing `make test`

[truatpasteurdotfr]

Version of Singularity:

2.4-dist (built from source to make sure that it's not a abuild issue)

Expected behavior

make test passes

Actual behavior

 + sudo singularity build /tmp/stest.HbeeLL/container /tmp/stest.HbeeLL/container2               (retval=0) OK
 + singularity exec /tmp/stest.HbeeLL/container true                                             (retval=0) OK
 + singularity exec /tmp/stest.HbeeLL/container false                                            (retval=1) OK
 + singularity exec /tmp/stest.HbeeLL/container test -f /.singularity.d/runscript                (retval=0) OK
 + singularity exec /tmp/stest.HbeeLL/container test -f /.singularity.d/env/01-base.sh           (retval=0) OK
 + singularity exec /tmp/stest.HbeeLL/container test -f /.singularity.d/actions/shell            (retval=0) OK
 + singularity exec /tmp/stest.HbeeLL/container test -f /.singularity.d/actions/exec             (retval=0) OK
 + singularity exec /tmp/stest.HbeeLL/container test -f /.singularity.d/actions/run              (retval=0) OK
 + singularity exec /tmp/stest.HbeeLL/container test -L /environment                             (retval=0) OK
 + singularity exec /tmp/stest.HbeeLL/container test -L /singularity                             (retval=0) OK
 + singularity build /tmp/stest.HbeeLL/container shub://GodloveD/busybox                       (retval=255) ERROR
Cache folder set to /home/tru/.singularity/shub
Progress |===================================| 100.0% gzip: /home/tru/.singularity/shub/GodloveD-busybox-master.simg: No such file or directory
ERROR Error extracting image, cleaning up.

Steps to reproduce behavior

./configure --prefix/opt/singularity && make && sudo make install && sudo ln -s /opt/singularity/bin/singularity /usr/bin/singularity && make test

[truatpasteurdotfr]

additionnal data point: running from the CLI with debug:

alpine:~/singularity-build/2.4.0/singularity-2.4$  singularity  -d  build /tmp/stest.HbeeLL/container shub://GodloveD/busybox     
Enabling debugging
Ending argument loop
Singularity version: 2.4-dist
Exec'ing: /opt/singularity/libexec/singularity/cli/build.exec
VERBOSE2 SINGULARITY_COMMAND_ASIS found as False
VERBOSE2 SINGULARITY_ROOTFS found as /tmp/.singularity-build.lhbCld
VERBOSE2 SINGULARITY_METADATA_FOLDER found as /tmp/.singularity-build.lhbCld/.singularity.d
VERBOSE2 SINGULARITY_FIX_PERMS found as False
VERBOSE2 SINGULARITY_COLORIZE not defined (None)
VERBOSE2 SINGULARITY_DISABLE_CACHE found as False
VERBOSE2 SINGULARITY_CACHEDIR found as /home/tru/.singularity
VERBOSE2 REGISTRY not defined (None)
VERBOSE2 NAMESPACE not defined (None)
VERBOSE2 SINGULARITY_DOCKER_ARCHITECTURE found as amd64
VERBOSE2 SINGULARITY_DOCKER_OS found as linux
VERBOSE2 SINGULARITY_ENVIRONMENT found as /tmp/.singularity-build.lhbCld/.singularity.d/env/90-environment.sh
VERBOSE2 SINGULARITY_RUNSCRIPT found as /tmp/.singularity-build.lhbCld/singularity
VERBOSE2 SINGULARITY_TESTFILE found as /tmp/.singularity-build.lhbCld/.singularity.d/test
VERBOSE2 SINGULARITY_DEFFILE found as /tmp/.singularity-build.lhbCld/.singularity.d/Singularity
VERBOSE2 SINGULARITY_HELPFILE found as /tmp/.singularity-build.lhbCld/.singularity.d/runscript.help
VERBOSE2 SINGULARITY_ENVBASE found as /tmp/.singularity-build.lhbCld/.singularity.d/env
VERBOSE2 SINGULARITY_LABELFILE found as /tmp/.singularity-build.lhbCld/.singularity.d/labels.json
VERBOSE2 SINGULARITY_INCLUDECMD found as False
VERBOSE2 SINGULARITY_NOHTTPS found as False
VERBOSE2 SINGULARITY_PULLFOLDER found as /home/tru/singularity-build/2.4.0/singularity-2.4
VERBOSE2 SHUB_NAMEBYHASH not defined (None)
VERBOSE2 SHUB_NAMEBYCOMMIT not defined (None)
VERBOSE2 SHUB_CONTAINERNAME not defined (None)
VERBOSE2 SINGULARITY_CONTENTS found as /tmp/.singularity-layerfile.ojpIbc
VERBOSE2 SINGULARITY_PYTHREADS found as 9
DEBUG 
*** STARTING SINGULARITY PYTHON PULL ****
VERBOSE2 SINGULARITY_CONTAINER found as shub://GodloveD/busybox
VERBOSE2 SINGULARITY_PULLFOLDER not defined (None)
DEBUG Found uri shub://
DEBUG Headers found: Content-Type,Accept
DEBUG GET https://singularity-hub.org/api/container/GodloveD/busybox:latest
Cache folder set to /home/tru/.singularity/shub
DEBUG Pull folder set to /home/tru/.singularity/shub
DEBUG Pulling to /home/tru/.singularity/shub/GodloveD-busybox-master.simg
DEBUG GET (stream) https://www.googleapis.com/download/storage/v1/b/singularity-hub-regional/o/singularity-hub-regional%2Fgithub.com%2FGodloveD%2Fbusybox%2F1d348bdd8e7123e1012e5ca52c028e0f19e18bef%2Fd388a8f6db8652c36b3c80c0ffe39f94.img.gz?generation=1504713234159841&alt=media
Progress |===================================| 100.0% 
DEBUG Found GZIP image
VERBOSE2 Running command gzip -d -f /home/tru/.singularity/shub/GodloveD-busybox-master.simg with subprocess
gzip: /home/tru/.singularity/shub/GodloveD-busybox-master.simg: No such file or directory
ERROR Error extracting image, cleaning up.
VERBOSE3 Cleaning up /home/tru/.singularity/shub/GodloveD-busybox-master.simg.gz
DEBUG Writing Singularity Hub image path to /tmp/.singularity-layerfile.ojpIbc
VERBOSE2 Writing file /tmp/.singularity-layerfile.ojpIbc with mode w.
ERROR: Unknown container build definition format: /home/tru/.singularity/shub/GodloveD-busybox-master.simg
ABORT: Aborting with RETVAL=255
Cleaning up...

fetching the file is not an issue:

alpine:~/singularity-build/2.4.0/singularity-2.4$ curl 'https://www.googleapis.com/download/storage/v1/b/singularity-hub-regional/o/singularity-hub-regional%2Fgithub.com%2FGodloveD%2Fbusybox%2F1d348bdd8e7123e1012e5ca52c028e0f19e18bef%2Fd388a8f6db8652c36b3c80c0ffe39f94.img.gz?generation=1504713234159841&alt=media'| md5sum
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1180k  100 1180k    0     0  1180k      0  0:00:01 --:--:--  0:00:01 2100k
9922651ee7b772d8f8e108bc3ad618c3  -
alpine:~/singularity-build/2.4.0/singularity-2.4$ wget 'https://www.googleapis.com/download/storage/v1/b/singularity-hub-regional/o/singularity-hub-regional%2Fgithub.com%2FGodloveD%2Fbusybox%2F1d348bdd8e7123e1012e5ca52c028e0f19e18bef%2Fd388a8f6db8652c36b3c80c0ffe39f94.img.gz?generation=1504713234159841&alt=media' -O - | md5sum 
--2017-10-12 08:20:30--  https://www.googleapis.com/download/storage/v1/b/singularity-hub-regional/o/singularity-hub-regional%2Fgithub.com%2FGodloveD%2Fbusybox%2F1d348bdd8e7123e1012e5ca52c028e0f19e18bef%2Fd388a8f6db8652c36b3c80c0ffe39f94.img.gz?generation=1504713234159841&alt=media
Resolving www.googleapis.com... 216.58.209.234, 216.58.213.170, 216.58.201.234, ...
Connecting to www.googleapis.com|216.58.209.234|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1208794 (1.2M) [text/plain]
Saving to: 'STDOUT'

-                                                    100%[======================================================================================================================>]   1.15M  --.-KB/s    in 0.07s   

2017-10-12 08:20:31 (15.4 MB/s) - written to stdout [1208794/1208794]

9922651ee7b772d8f8e108bc3ad618c3  -

[vsoch]

it looks like it's trying to fetch the file and name it without the gz extension, and then extract that file with (which doesn't exist).

[vsoch]

This is the line that is (it seems to be) failing to work:

https://github.com/singularityware/singularity/blob/development/libexec/python/shub/api.py#L176

[vsoch]

This is an easy fix. This part:

        if image_type == "GZIP" or extract is True:

            if not image_file.endswith('.gz'):
                os.rename(image_file, "%s.gz" % image_file)

needs to be this:

        if image_type == "GZIP" or extract is True:

            if not image_file.endswith('.gz'):
                os.rename(image_file, "%s.gz" % image_file)
                image_file = "%s.gz" % image_file

[truatpasteurdotfr]

thanks, that fixes the 1st error :)

Now the test fails further down:

################################################################################
Standard Integration Format (SCI-F) Apps bootstrap tests (script: 23-bootstrap_apps.sh, testdir: /tmp/stest.OgBaJn)

 + grep ubuntu:14.04 ../examples/apps/Singularity                                                (retval=0) OK
 + cp ../examples/apps/Singularity /tmp/stest.OgBaJn/example.def                                 (retval=0) OK
 + singularity create -F -s 568 /tmp/stest.OgBaJn/container.img                                  (retval=0) OK
 + sudo singularity bootstrap /tmp/stest.OgBaJn/container.img /tmp/stest.OgBaJn/example.def      (retval=0) OK
 + singularity exec /tmp/stest.OgBaJn/container.img true                                         (retval=0) OK
 + singularity exec /tmp/stest.OgBaJn/container.img /bin/true                                    (retval=0) OK
 + singularity exec /tmp/stest.OgBaJn/container.img false                                        (retval=1) OK
 + singularity exec /tmp/stest.OgBaJn/container.img /bin/false                                   (retval=1) OK
 + singularity exec /tmp/stest.OgBaJn/container.img test -d /scif                                (retval=0) OK
 + singularity exec /tmp/stest.OgBaJn/container.img test -d /scif/apps                           (retval=0) OK
 + singularity exec /tmp/stest.OgBaJn/container.img test -d /scif/data                           (retval=0) OK
 + singularity exec /tmp/stest.OgBaJn/container.img test -d /scif/apps/foo                       (retval=0) OK
 + singularity exec /tmp/stest.OgBaJn/container.img test -d /scif/apps/bar                       (retval=0) OK
 + singularity exec /tmp/stest.OgBaJn/container.img test -f /scif/apps/foo/filefoo.exec          (retval=1) ERROR
Full output in: /tmp/stest.OgBaJn
ERROR: Failed running test: 23-bootstrap_apps.sh
make: *** [Makefile:820: test] Error 1
alpine:~/singularity-build/2.4.0/singularity-2.4$ singularity exec /tmp/stest.OgBaJn/container.img test -f /scif/apps/foo/filefoo.exec      
alpine:~/singularity-build/2.4.0/singularity-2.4$ echo $?
1
alpine:~/singularity-build/2.4.0/singularity-2.4$ singularity exec /tmp/stest.OgBaJn/container.img /bin/ls -l /scif/apps/foo/filefoo.exec      
/bin/ls: cannot access /scif/apps/foo/filefoo.exec: No such file or directory
alpine:~/singularity-build/2.4.0/singularity-2.4$ echo $?
2

if /scif/apps/foo/filefoo.exec does not exists, this error code should be expected, imho. But tests/23-bootstrap_apps.sh has:

# Testing folder organization
stest 0 singularity exec "$CONTAINER" test -d "/scif"
stest 0 singularity exec "$CONTAINER" test -d "/scif/apps"
stest 0 singularity exec "$CONTAINER" test -d "/scif/data"
stest 0 singularity exec "$CONTAINER" test -d "/scif/apps/foo"
stest 0 singularity exec "$CONTAINER" test -d "/scif/apps/bar"
stest 0 singularity exec "$CONTAINER" test -f "/scif/apps/foo/filefoo.exec"
stest 0 singularity exec "$CONTAINER" test -f "/scif/apps/bar/filebar.exec"
stest 0 singularity exec "$CONTAINER" test -d "/scif/data/foo/output"
stest 0 singularity exec "$CONTAINER" test -d "/scif/data/foo/input"

[vsoch]

sorry can you be more explicit? Are you saying that the file should exist and the test is wrong, or it shouldn't exist and the test is right? To clarify, the file should exist in the container:

%appinstall foo

    echo "INSTALLING FOO"
    touch filefoo.exec

[truatpasteurdotfr]

cannot access /scif/apps/foo/filefoo.exec: No such file or directory seems to indicate that the file does not exist in my tested container, so the test is expected to fail and tested as such in the test script.

[truatpasteurdotfr]

alpine:~/singularity-build/2.4.0/singularity-2.4$ singularity exec /tmp/stest.OgBaJn/container.img find /scif -ls
 70349    4 drwxrwxr-x   4 root     root         4096 Oct 12 09:53 /scif
 70365    4 drwxrwxr-x   4 root     root         4096 Oct 12 09:53 /scif/apps
 70366    4 drwxrwxr-x   5 root     root         4096 Oct 12 09:53 /scif/apps/bar
 70367    4 drwxrwxr-x   2 root     root         4096 Oct 12 09:53 /scif/apps/bar/lib
 70368    4 drwxrwxr-x   2 root     root         4096 Oct 12 09:53 /scif/apps/bar/bin
 70369    4 drwxrwxr-x   3 root     root         4096 Oct 12 09:53 /scif/apps/bar/scif
  7296    4 -rw-rw-r--   1 root     root           59 Oct 12 09:53 /scif/apps/bar/scif/Singularity
  7297    4 -rw-rw-r--   1 root     root           72 Oct 12 09:53 /scif/apps/bar/scif/labels.json
 70372    4 drwxrwxr-x   2 root     root         4096 Oct 12 09:53 /scif/apps/bar/scif/env
  7295    4 -rw-rw-r--   1 root     root          445 Oct 12 09:53 /scif/apps/bar/scif/env/01-base.sh
 70374    4 drwxrwxr-x   5 root     root         4096 Oct 12 09:53 /scif/apps/foo
 70375    4 drwxrwxr-x   2 root     root         4096 Oct 12 09:53 /scif/apps/foo/lib
 70376    4 drwxrwxr-x   2 root     root         4096 Oct 12 09:53 /scif/apps/foo/bin
 70377    4 drwxrwxr-x   3 root     root         4096 Oct 12 09:53 /scif/apps/foo/scif
  7288    4 -rw-rw-r--   1 root     root           59 Oct 12 09:53 /scif/apps/foo/scif/Singularity
  7289    4 -rw-rw-r--   1 root     root           72 Oct 12 09:53 /scif/apps/foo/scif/labels.json
 70380    4 drwxrwxr-x   2 root     root         4096 Oct 12 09:53 /scif/apps/foo/scif/env
  7286    4 -rw-rw-r--   1 root     root          445 Oct 12 09:53 /scif/apps/foo/scif/env/01-base.sh
  7287    0 -rw-rw-r--   1 root     root            0 Oct 12 09:53 /scif/apps/foo/scif/env/90-environment.sh
 70383    4 drwxrwxr-x   4 root     root         4096 Oct 12 09:53 /scif/data
 70384    4 drwxrwxr-x   4 root     root         4096 Oct 12 09:53 /scif/data/bar
 70385    4 drwxrwxr-x   2 root     root         4096 Oct 12 09:53 /scif/data/bar/output
 70386    4 drwxrwxr-x   2 root     root         4096 Oct 12 09:53 /scif/data/bar/input
 70387    4 drwxrwxr-x   4 root     root         4096 Oct 12 09:53 /scif/data/foo
 70388    4 drwxrwxr-x   2 root     root         4096 Oct 12 09:53 /scif/data/foo/output
 70389    4 drwxrwxr-x   2 root     root         4096 Oct 12 09:53 /scif/data/foo/input

[truatpasteurdotfr]

Something is wrong in that container... neither filefoo.exec nor filebar.exec are there.

So your tests are good, they caught something :P

[vsoch]

Does alpine linux have the touch command?

[truatpasteurdotfr]

yes as /bin/touch, further more I can touch outside or inside the container:

alpine:~/singularity-build/2.4.0/singularity-2.4$ singularity exec /tmp/stest.INMcGC/container.img  ls -l /bin/touch
-rwxr-xr-x 1 root root 60224 Mar 10  2016 /bin/touch
alpine:~/singularity-build/2.4.0/singularity-2.4$ singularity exec /tmp/stest.OgBaJn/container.img /bin/touch /tmp/AAAA
alpine:~/singularity-build/2.4.0/singularity-2.4$ ls -l /tmp/AAAA
-rw-r--r--    1 tru      Bis              0 Oct 12 12:20 /tmp/AAAA
alpine:~/singularity-build/2.4.0/singularity-2.4$ sudo singularity exec -w /tmp/stest.OgBaJn/container.img /bin/touch /scif/data/foo/COUCOU
alpine:~/singularity-build/2.4.0/singularity-2.4$ singularity exec /tmp/stest.OgBaJn/container.img ls -ld /scif/data/foo/COUCOU
-rw-r--r-- 1 root root 0 Oct 12 12:21 /scif/data/foo/COUCOU

[truatpasteurdotfr]

no additionnal info if I just re-run that test

alpine:~/singularity-build/2.4.0/singularity-2.4$ sh -x  test.sh 23-bootstrap_apps.sh 
+ prefix=/opt/singularity
+ exec_prefix=/opt/singularity
+ libexecdir=/opt/singularity/libexec
+ sysconfdir=/opt/singularity/etc
+ localstatedir=/opt/singularity/var
+ bindir=/opt/singularity/bin
+ SINGULARITY_OVERLAY_FS=1
+ SINGULARITY_USER_NS=
+ SINGULARITY_libexecdir=/opt/singularity/libexec
+ SINGULARITY_sysconfdir=/opt/singularity/etc
+ SINGULARITY_localstatedir=/opt/singularity/var
+ SINGULARITY_PATH=/opt/singularity/bin
+ export SINGULARITY_libexecdir SINGULARITY_sysconfdir SINGULARITY_localstatedir SINGULARITY_PATH SINGULARITY_OVERLAY_FS SINGULARITY_USER_NS
+ [ -z  ]
+ /bin/echo Building/Installing Singularity to temporary directory
Building/Installing Singularity to temporary directory
+ /bin/echo Reinvoking in a clean shell
Reinvoking in a clean shell
+ sleep 1
+ exec env -i CLEAN_SHELL=1 PATH=/opt/singularity/bin:/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin:/usr/local/sbin bash test.sh 23-bootstrap_apps.sh

################################################################################
Standard Integration Format (SCI-F) Apps bootstrap tests (script: 23-bootstrap_apps.sh, testdir: /tmp/stest.iMLiCL)

 + grep ubuntu:14.04 ../examples/apps/Singularity                                                (retval=0) OK
 + cp ../examples/apps/Singularity /tmp/stest.iMLiCL/example.def                                 (retval=0) OK
 + singularity create -F -s 568 /tmp/stest.iMLiCL/container.img                                  (retval=0) OK
 + sudo singularity bootstrap /tmp/stest.iMLiCL/container.img /tmp/stest.iMLiCL/example.def      (retval=0) OK
 + singularity exec /tmp/stest.iMLiCL/container.img true                                         (retval=0) OK
 + singularity exec /tmp/stest.iMLiCL/container.img /bin/true                                    (retval=0) OK
 + singularity exec /tmp/stest.iMLiCL/container.img false                                        (retval=1) OK
 + singularity exec /tmp/stest.iMLiCL/container.img /bin/false                                   (retval=1) OK
 + singularity exec /tmp/stest.iMLiCL/container.img test -d /scif                                (retval=0) OK
 + singularity exec /tmp/stest.iMLiCL/container.img test -d /scif/apps                           (retval=0) OK
 + singularity exec /tmp/stest.iMLiCL/container.img test -d /scif/data                           (retval=0) OK
 + singularity exec /tmp/stest.iMLiCL/container.img test -d /scif/apps/foo                       (retval=0) OK
 + singularity exec /tmp/stest.iMLiCL/container.img test -d /scif/apps/bar                       (retval=0) OK
 + singularity exec /tmp/stest.iMLiCL/container.img test -f /scif/apps/foo/filefoo.exec          (retval=1) ERROR
Full output in: /tmp/stest.iMLiCL
ERROR: Failed running test: 23-bootstrap_apps.sh

[truatpasteurdotfr]

alpine:~/singularity-build/2.4.0/singularity-2.4$ sudo singularity -d bootstrap /tmp/stest.iMLiCL/container.img /tmp/stest.iMLiCL/example.def

yields some errors with:

Enabling debugging
Ending argument loop
Singularity version: 2.4-dist
Exec'ing: /opt/singularity/libexec/singularity/cli/bootstrap.exec
WARNING: The bootstrap command is deprecated and will be removed in a future release.
WARNING: Use the build command like so:
WARNING: singularity build /tmp/stest.iMLiCL/container.img /tmp/stest.iMLiCL/example.def
Building into existing container: /tmp/stest.iMLiCL/container.img
Using container recipe deffile: /tmp/stest.iMLiCL/example.def
Sanitizing environment
Adding base Singularity environment to container
User defined %runscript found! Taking priority.
+ echo SETUP
SETUP
Running post scriptlet
+ echo POST
POST
Adding runscript
foo has help section
grep: unrecognized option: quiet
BusyBox v1.26.2 (2017-10-04 13:37:41 GMT) multi-call binary.

Usage: grep [-HhnlLoqvsriwFE] [-m N] [-A/B/C N] PATTERN/-e PATTERN.../-f FILE [FILE]...

Search for PATTERN in FILEs (or stdin)

    -H  Add 'filename:' prefix
    -h  Do not add 'filename:' prefix
    -n  Add 'line_no:' prefix
    -l  Show only names of files that match
    -L  Show only names of files that don't match
    -c  Show only count of matching lines
    -o  Show only the matching part of line
    -q  Quiet. Return 0 if PATTERN is found, 1 otherwise
    -v  Select non-matching lines
    -s  Suppress open and read errors
    -r  Recurse
    -i  Ignore case
    -w  Match whole words only
    -x  Match whole lines only
    -F  PATTERN is a literal (not regexp)
    -E  PATTERN is an extended regexp
    -m N    Match up to N times per file
    -A N    Print N lines of trailing context
    -B N    Print N lines of leading context
    -C N    Same as '-A N -B N'
    -e PTRN Pattern to match
    -f FILE Read pattern from file
grep: unrecognized option: quiet

there are grep --quiet floatting in ./libexec/bootstrap-scripts/functions that are not portable :D

[truatpasteurdotfr]

once the 2 lines fixed:

alpine:~/singularity-build/2.4.0/singularity-2.4$ sudo singularity -d bootstrap  /tmp/stest.iMLiCL/container.img /tmp/stest.iMLiCL/example.def
Enabling debugging
Ending argument loop
Singularity version: 2.4-dist
Exec'ing: /opt/singularity/libexec/singularity/cli/bootstrap.exec
WARNING: The bootstrap command is deprecated and will be removed in a future release.
WARNING: Use the build command like so:
WARNING: singularity build /tmp/stest.iMLiCL/container.img /tmp/stest.iMLiCL/example.def
Building into existing container: /tmp/stest.iMLiCL/container.img
Using container recipe deffile: /tmp/stest.iMLiCL/example.def
Sanitizing environment
Adding base Singularity environment to container
User defined %runscript found! Taking priority.
+ echo SETUP
SETUP
Running post scriptlet
+ echo POST
POST
Adding runscript
foo has help section
foo has runscript definition
ERROR SINGULARITY_APP_SIZE found in /opt/singularity/var/singularity/mnt/container/scif/apps/foo/scif/labels.json and overwrite set to False.
ERROR SINGULARITY_APP_NAME found in /opt/singularity/var/singularity/mnt/container/scif/apps/foo/scif/labels.json and overwrite set to False.
Adding custom environment to foo
Adding labels to foo
Adding labels to bar
Installing foo
+ cd /scif/apps/foo
+ echo INSTALLING FOO
INSTALLING FOO
+ touch filefoo.exec
Installing bar
+ cd /scif/apps/bar
+ echo INSTALLING BAR
INSTALLING BAR
+ touch filebar.exec
Found an existing definition file
Adding a bootstrap_history directory
Finalizing Singularity container
date: unrecognized option: rfc-3339=seconds
BusyBox v1.26.2 (2017-10-04 13:37:41 GMT) multi-call binary.

Usage: date [OPTIONS] [+FMT] [TIME]

Display time (using +FMT), or set time

    [-s,--set] TIME Set time to TIME
    -u,--utc    Work in UTC (don't convert to local time)
    -R,--rfc-2822   Output RFC-2822 compliant date string
    -I[SPEC]    Output ISO-8601 compliant date string
            SPEC='date' (default) for date only,
            'hours', 'minutes', or 'seconds' for date and
            time to the indicated precision
    -r,--reference FILE Display last modification time of FILE
    -d,--date TIME  Display TIME, not 'now'
    -D FMT      Use FMT for -d TIME conversion

Recognized TIME formats:
    hh:mm[:ss]
    [YYYY.]MM.DD-hh:mm[:ss]
    YYYY-MM-DD hh:mm[:ss]
    [[[[[YY]YY]MM]DD]hh]mm[.ss]
    'date TIME' form accepts MMDDhhmm[[YY]YY][.ss] instead
ERROR --key and --file and --value must be defined for ADD.
Calculating final size for metadata...
du: unrecognized option: apparent-size
BusyBox v1.26.2 (2017-10-04 13:37:41 GMT) multi-call binary.

Usage: du [-aHLdclsxhmk] [FILE]...

Summarize disk space used for each FILE and/or directory

    -a  Show file sizes too
    -L  Follow all symlinks
    -H  Follow symlinks on command line
    -d N    Limit output to directories (and files with -a) of depth < N
    -c  Show grand total
    -l  Count sizes many times if hard linked
    -s  Display only a total for each argument
    -x  Skip directories on different filesystems
    -h  Sizes in human readable format (e.g., 1K 243M 2G)
    -m  Sizes in megabytes
    -k  Sizes in kilobytes (default)
Skipping checks
Building Singularity image...
Singularity container built: /tmp/stest.iMLiCL/container.img
Cleaning up...

[vsoch]

so - we are good?

[truatpasteurdotfr]

alpine linux date does not have rfc-3339=seconds and du is missing apparent-size

[truatpasteurdotfr]

du -s is ok but I have no idea to fix the date formatting between coreutils and busybox. And that does not explain why the touched files are missing!

[truatpasteurdotfr]

date -R exist for both but that gives:

alpine:~/singularity-build/2.4.0/singularity-2.4$ date -R
Thu, 12 Oct 2017 12:41:32 +0000

[tru@elitebook840g3 ~]$ date -R
Thu, 12 Oct 2017 21:04:44 +0200

[truatpasteurdotfr]

alpine linux does not have the exclude=

BusyBox v1.26.2 (2017-10-04 13:37:41 GMT) multi-call binary.

Usage: du [-aHLdclsxhmk] [FILE]...

Summarize disk space used for each FILE and/or directory

    -a  Show file sizes too
    -L  Follow all symlinks
    -H  Follow symlinks on command line
    -d N    Limit output to directories (and files with -a) of depth < N
    -c  Show grand total
    -l  Count sizes many times if hard linked
    -s  Display only a total for each argument
    -x  Skip directories on different filesystems
    -h  Sizes in human readable format (e.g., 1K 243M 2G)
    -m  Sizes in megabytes
    -k  Sizes in kilobytes (default)

[truatpasteurdotfr]

I have "fixed" the du with:

message 1 "Calculating final size for metadata...\n"
IMAGE_SIZE=$(du -s -x $SINGULARITY_ROOTFS | cut -f 1)

and

eval $SINGULARITY_ADD_SCRIPT -f --key "org.label-schema.build-date" --value $(date -R | sed 's/ /_/') --file $SINGULARITY_LABELFILE

now I have:

alpine:~/singularity-build/2.4.0/singularity-2.4$ sh -x  test.sh 23-bootstrap_apps.sh 
+ prefix=/opt/singularity
+ exec_prefix=/opt/singularity
+ libexecdir=/opt/singularity/libexec
+ sysconfdir=/opt/singularity/etc
+ localstatedir=/opt/singularity/var
+ bindir=/opt/singularity/bin
+ SINGULARITY_OVERLAY_FS=1
+ SINGULARITY_USER_NS=
+ SINGULARITY_libexecdir=/opt/singularity/libexec
+ SINGULARITY_sysconfdir=/opt/singularity/etc
+ SINGULARITY_localstatedir=/opt/singularity/var
+ SINGULARITY_PATH=/opt/singularity/bin
+ export SINGULARITY_libexecdir SINGULARITY_sysconfdir SINGULARITY_localstatedir SINGULARITY_PATH SINGULARITY_OVERLAY_FS SINGULARITY_USER_NS
+ [ -z  ]
+ /bin/echo Building/Installing Singularity to temporary directory
Building/Installing Singularity to temporary directory
+ /bin/echo Reinvoking in a clean shell
Reinvoking in a clean shell
+ sleep 1
+ exec env -i CLEAN_SHELL=1 PATH=/opt/singularity/bin:/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin:/usr/local/sbin bash test.sh 23-bootstrap_apps.sh

################################################################################
Standard Integration Format (SCI-F) Apps bootstrap tests (script: 23-bootstrap_apps.sh, testdir: /tmp/stest.mIKgoJ)

 + grep ubuntu:14.04 ../examples/apps/Singularity                                                (retval=0) OK
 + cp ../examples/apps/Singularity /tmp/stest.mIKgoJ/example.def                                 (retval=0) OK
 + singularity create -F -s 568 /tmp/stest.mIKgoJ/container.img                                  (retval=0) OK
 + sudo singularity bootstrap /tmp/stest.mIKgoJ/container.img /tmp/stest.mIKgoJ/example.def      (retval=0) OK
 + singularity exec /tmp/stest.mIKgoJ/container.img true                                         (retval=0) OK
 + singularity exec /tmp/stest.mIKgoJ/container.img /bin/true                                    (retval=0) OK
 + singularity exec /tmp/stest.mIKgoJ/container.img false                                        (retval=1) OK
 + singularity exec /tmp/stest.mIKgoJ/container.img /bin/false                                   (retval=1) OK
 + singularity exec /tmp/stest.mIKgoJ/container.img test -d /scif                                (retval=0) OK
 + singularity exec /tmp/stest.mIKgoJ/container.img test -d /scif/apps                           (retval=0) OK
 + singularity exec /tmp/stest.mIKgoJ/container.img test -d /scif/data                           (retval=0) OK
 + singularity exec /tmp/stest.mIKgoJ/container.img test -d /scif/apps/foo                       (retval=0) OK
 + singularity exec /tmp/stest.mIKgoJ/container.img test -d /scif/apps/bar                       (retval=0) OK
 + singularity exec /tmp/stest.mIKgoJ/container.img test -f /scif/apps/foo/filefoo.exec          (retval=0) OK
 + singularity exec /tmp/stest.mIKgoJ/container.img test -f /scif/apps/bar/filebar.exec          (retval=0) OK
 + singularity exec /tmp/stest.mIKgoJ/container.img test -d /scif/data/foo/output                (retval=0) OK
 + singularity exec /tmp/stest.mIKgoJ/container.img test -d /scif/data/foo/input                 (retval=0) OK
 + singularity exec /tmp/stest.mIKgoJ/container.img test -d /scif/apps/foo/scif                  (retval=0) OK
 + singularity exec /tmp/stest.mIKgoJ/container.img test -d /scif/apps/foo/scif/env              (retval=0) OK
 + singularity exec /tmp/stest.mIKgoJ/container.img test -f /scif/apps/foo/scif/Singularity      (retval=0) OK
 + singularity exec /tmp/stest.mIKgoJ/container.img test -f /scif/apps/foo/scif/env/01-base.s    (retval=0) OK
 + singularity exec /tmp/stest.mIKgoJ/container.img test -f /scif/apps/foo/scif/labels.json      (retval=0) OK
 + singularity exec /tmp/stest.mIKgoJ/container.img test -f /scif/apps/foo/scif/runscript        (retval=0) OK
 + singularity exec /tmp/stest.mIKgoJ/container.img test -f /scif/apps/foo/scif/runscript.hel    (retval=0) OK
 + sh -c singularity help '/tmp/stest.mIKgoJ/container.img' | grep 'No runscript help is defi    (retval=0) OK
 + sh -c singularity help --app foo '/tmp/stest.mIKgoJ/container.img' | grep 'This is the hel    (retval=0) OK
 + sh -c singularity help --app bar '/tmp/stest.mIKgoJ/container.img' | grep 'No runscript he    (retval=0) OK
 + sh -c singularity apps '/tmp/stest.mIKgoJ/container.img' | grep 'foo'                         (retval=0) OK
 + sh -c singularity apps '/tmp/stest.mIKgoJ/container.img' | grep 'bar'                         (retval=0) OK
 + sh -c singularity inspect --app foo '/tmp/stest.mIKgoJ/container.img' | grep HELLOTHISIS      (retval=0) OK
 + sh -c singularity inspect --app foo '/tmp/stest.mIKgoJ/container.img' | grep foo              (retval=0) OK
 + sh -c singularity run --app foo '/tmp/stest.mIKgoJ/container.img' | grep 'RUNNING FOO'        (retval=0) OK
 + sh -c singularity run --app bar '/tmp/stest.mIKgoJ/container.img' | grep 'No Singularity r    (retval=0) OK

All tests passed

[dtrudg]

Closing as we believe this is fixed when #1050 was merged. Let us know if any issues in current Singularity. Thanks!