Issues using Docker image with singularity

[RDoerfel]

Hello, unfortunately I have issues using the docker image. Using run, it can't find entripoint.py, and when using exec, there are issues sourcing the .bashrc script.

Here is what I do:

export SINGULARITY_TMPDIR=<path_to_directory_of_choice>

singularity pull docker mcc.sif docker://ghcr.io/wwu-mmll/mccqrnn_docker

singularity run \
                --cleanenv \
                --bind <my_input_dir>:/input \
                --bind <my_output_dir>:/output \
                <path_to_image>/mcc.sif

When running this, I get the following error:

/users/rubendorfel/.bashrc: line 30: /usr/local/etc/.profile: No such file or directory
python: can't open file '/users/rubendorfel/entrypoint.py': [Errno 2] No such file or directory

Alternatively I tried to set the home-directory manually and point towards the .bashrc script as it couldn't find this before.

singularity exec \
            --cleanenv \
            --bind /indirect/student/rubendorfel/brain-age-trt/3Party/mcc/input:/input \
            --bind /indirect/student/rubendorfel/brain-age-trt/3Party/mcc/input:/output \
            /indirect/proc_data1/brainage/src/singularity_utils/singularity-sif/mcc.sif \
            source /users/rubendorfel/.bashrc && python /mcc/entrypoint.py 

Still no succes here. I can execute python/mcc/entrypoint.py, but then the conda environment is not activated. when sourcing /users/rubendorfel/.bashrc it complains about not being able to find /usr/local/etc/.profile

[jernsting]

I have not tested the docker container with singulartiy. Could you try using docker for one example and test if this runs? It might be some singulartity related problem.

[RDoerfel]

This is unfortunately not possible as I do not have admin rights on the cluster

[jernsting]

Could you start the container interactively, locate the entrypoint-script and run it manually (also only for one example). I suspect that singularity is trying to invoke the entrypoint in the wrong directory. When the container is running, there should be an /mcc directory containing the script. But it seems like singularity is trying to find the entrypoint.py in your home directory.

[RDoerfel]

Yes this is what happens (among other things I guess). When executing the entrypoint.py script I get the following:

  File "/mcc/entrypoint.py", line 5, in <module>
    import numpy as np
ModuleNotFoundError: No module named 'numpy'

I guess the problem is that the conda environment is not properly installed / activated as source .bashrc is not working.

[jernsting]

Yes. It works with Docker, but not with Singularity. Unfortunately, I don't have capacity to fix this problem.... So if you want to use Singularity, you have to find a solution yourself, sorry.

If you find a good solution, I would really appreciate feedback to make the container usable with Singularity :)

[RDoerfel]

Just tried to run and source everything within the image and it seems to work.

I'll see if I can get it up and running. Thanks for your quick replies so far :)

[RDoerfel]

It seems like the problem is somewhat related to activating the 'conda' environment as described here: https://pythonspeed.com/articles/activate-conda-dockerfile/. I'll try to build some local copies and see if I can fix it.

[RDoerfel]

So I got it up and running on a local machine using Docker. Hence I guess it is singularity related. I'll see how I can solve it.

[RDoerfel]

Ok so an issue is that Docker and Singularity treat the shortcut ~/ differently. While on Docker the appropriate .bashrc is in /root (where I do not have permissions), Singularity looks in the home which is mirrored from the host system. Hence, two different .bashrc files are sourced.

One solution could be to directly activate the conda environment as suggested here, instead of sourcing .bashrc in the Dockerfile.

. /opt/conda/etc/profile.d/conda.sh
conda activate myenv