Bug in singularity-compose.yml example

[serbinsh]

Following instructions here: https://singularityhub.github.io/singularity-compose/#/?id=singularity-compose

Installing collected packages: pip

  Found existing installation: pip 19.1.1

    Uninstalling pip-19.1.1:

      Successfully uninstalled pip-19.1.1

Successfully installed pip-20.2.2

INFO:    Starting build...
Getting image source signatures
Copying blob sha256:c87736221ed0bcaa60b8e92a19bec2284899ef89226f2a07968677cf59e637a4
Copying blob sha256:c3f51b0d0765f7359fcc1b8d67887b1461c51dfb04b68ad54676a592af48cfdc
Copying blob sha256:a65abebf548020f86dedbaeebd52cabe9c517767097a538b2501f9b7e8b185f5
Copying blob sha256:6628a73c2c8584cad0284cc99cd31d3643520557539f03f360912436cd59ab33
Copying blob sha256:b49d22f17d2f47ff079f657c80e3249f956dfa0ada4d88806dac1d23b82ebcb0
Copying config sha256:1ab1e8afa891e8ec6b6ee28f28d1b97fcd619fe1b4f926a31523b43f558529e8
Writing manifest to image destination
Storing signatures
2020/08/27 10:35:06  info unpack layer: sha256:c87736221ed0bcaa60b8e92a19bec2284899ef89226f2a07968677cf59e637a4
2020/08/27 10:35:06  info unpack layer: sha256:c3f51b0d0765f7359fcc1b8d67887b1461c51dfb04b68ad54676a592af48cfdc
2020/08/27 10:35:06  info unpack layer: sha256:a65abebf548020f86dedbaeebd52cabe9c517767097a538b2501f9b7e8b185f5
2020/08/27 10:35:10  info unpack layer: sha256:6628a73c2c8584cad0284cc99cd31d3643520557539f03f360912436cd59ab33
2020/08/27 10:35:10  info unpack layer: sha256:b49d22f17d2f47ff079f657c80e3249f956dfa0ada4d88806dac1d23b82ebcb0
INFO:    Running setup scriptlet
+ mkdir -p /tmp/rootfs-7f1a95ca-e872-11ea-9bd0-c81f66ea555a/code
INFO:    Copying requirements.txt to /tmp/rootfs-7f1a95ca-e872-11ea-9bd0-c81f66ea555a/tmp/requirements.txt
INFO:    Copying manage.py to /tmp/rootfs-7f1a95ca-e872-11ea-9bd0-c81f66ea555a/code/manage.py
INFO:    Copying run_uwsgi.sh to /tmp/rootfs-7f1a95ca-e872-11ea-9bd0-c81f66ea555a/code/run_uwsgi.sh
INFO:    Copying upload to /tmp/rootfs-7f1a95ca-e872-11ea-9bd0-c81f66ea555a/code/upload
INFO:    Copying uwsgi.ini to /tmp/rootfs-7f1a95ca-e872-11ea-9bd0-c81f66ea555a/code/uwsgi.ini
INFO:    Copying nginx/install.sh to /tmp/rootfs-7f1a95ca-e872-11ea-9bd0-c81f66ea555a/usr/src/install.sh
INFO:    Copying nginx/nginx.key to /tmp/rootfs-7f1a95ca-e872-11ea-9bd0-c81f66ea555a/usr/src/nginx.key
INFO:    Copying nginx/nginx.conf to /tmp/rootfs-7f1a95ca-e872-11ea-9bd0-c81f66ea555a/etc/nginx/nginx.conf
INFO:    Copying nginx/nginx.vh.default.conf to /tmp/rootfs-7f1a95ca-e872-11ea-9bd0-c81f66ea555a/etc/nginx/conf.d/default.conf
INFO:    Running post scriptlet
+ apk update
+ apk add --no-cache --virtual .build-deps sqlite libc-dev linux-headers sqlite-dev cmake gcc openssl wget git vim
+ pip install --upgrade pip
+ pip install -r /tmp/requirements.txt
ERROR: Could not open requirements file: [Errno 2] No such file or directory: '/tmp/requirements.txt'
FATAL:   While performing build: while running engine: exit status 1

WARNING Issue building container, try: sudo singularity build app.sif Singularity

[sserbin@modex singularity-compose-simple]$ pip install -r /tmp/requirements.txt
Defaulting to user installation because normal site-packages is not writeable
ERROR: Could not open requirements file: [Errno 2] No such file or directory: '/tmp/requirements.txt

if I then run

touch /tmp/requirements.txt

Build finishing fine.

Also FYI - I tried to find a discussion of this but I installed singularity follow the standard build instructions. However no matter what I tried when singularity-compose ran for this example and used sudo, root would not find singularity on the path. I had to edit the sudoers file and specifically

Defaults    secure_path = /sbin:/bin:/usr/sbin:/usr/bin:/data/software/singularity/3.6.2/bin

to point to my shared module install directory.

This is perhaps just specific to me but may be helpful to someone else in the future

Running on CentOS 8

[serbinsh]

Just to confirm I was able to get it running after all of this

[sserbin@modex singularity-compose-simple]$ singularity-compose up
[sserbin@modex singularity-compose-simple]$ singularity-compose ps
INSTANCES  NAME PID     IMAGE
1           app 2722056 app.sif

[vsoch]

I think your issue is with the example repository https://github.com/singularityhub/singularity-compose-simple and not the code here. I'll transfer it.

[vsoch]

The requirements.txt should be in the container - it could be that now that /tmp is bound to the host, it doesn't persist there. But I don't see an error with copying the file! Could you please try copying the file (this line https://github.com/singularityhub/singularity-compose-simple/blob/master/app/Singularity#L13) to somewhere else (e.g., /opt/requirements.txt) and see if that works?

[vsoch]

For the install of singularity issue, you'd want to post on https://github.com/hpcng/singularity/issues. This is unrelated to singularity-compose.

[serbinsh]

@vsoch thanks for the feedback! FYI - I recloned the example, edited app/Singularity to replace /tmp/requirements.txt with /opt/requirements.txt and re-rand singularity-compose build. This time no errors!

[vsoch]

Great! So I think using /tmp used to work, but now I think it doesn't! I'll put in a quick PR to change this to /opt. Thanks for the catch!

[serbinsh]

Ok as for the other issue, I noticed this when using singularity-compose when running this example. However, agreed I should probably move that issue to https://github.com/hpcng/singularity/issues because the problem was that when run as sudo (not by me but singularity-compose runs with sudo to config web) it was unable to find "singularity", the binary until i edited that file.

[serbinsh]

@vsoch no worries. Perhaps just an issue on my end or CentOS but I don't see why /opt wouldn't work in general....

[vsoch]

Please take a look! https://github.com/singularityhub/singularity-compose-simple/pull/2 We will merge when you give the A-OK.

To be clear, opt works fine! It's copying and then using in /tmp that seems to be the issue.

[serbinsh]

OK, so I

git clone git@github.com:singularityhub/singularity-compose-simple.git
git checkout -b opt origin/updating-tmp-opt
singularity-compose build
singularity-compose up

no errors

[vsoch]

Thank you!