search

apptainer / singularity

Singularity has been renamed to Apptainer as part of us moving the project to the Linux Foundation. This repo has been persisted as a snapshot right before the changes.
https://github.com/apptainer/apptainer
Other
2.52k stars 424 forks source link

Simple query: how to mount a host file to container output file #5419

Closed frankligy closed 4 years ago

frankligy commented 4 years ago

Query

To put it simple, I just hope to know how to bind/mount a host folder to a folder in container system if the software I am gonna run in the container have an output argument? In another word, my container would output a file, how to redirect it out to my host file system?

My concrete question is as below, much appreciated for any help!

Version of Singularity:

3.1.0

Expected behavior

So I am running a software that the author only provided the instruction of how to use docker to run as below:

docker run -v /path/to/data/folder:/data/ -t fred2/optitype -i input1 [input2] (-r|-d) -o /data/

From my understanding, they mounted a docker host folder(/path/to/data/folder) before the colon to the container folder(/data/) after the colon. As we can see, this container folder contains the outputs from the software (-o argument).

So I am trying to run it using Singularity since our cluster doesn't have docker available, I first build the .sif from docker hub using:

singularity build my_software.sif docker://fred2/optitype

Everything works well, then I started to run:

singularity run -B /path/to/data/folder:/data/ my_software.sif -i input1 [input2] (-r|-d) -o /data/

I expected it will work.

Actual behavior

It popped an error: IOError: [Errno 2] could not open alignment file result/2020_07_08_14_33_58/2020_07_08_14_33_58_1.bam: No such file or directory

Of course, it is related to the software itself, I asked the original author of the software, they believe there's something wrong with the mounting options however they don't use singularity, so I want to ask here what would be the right way to mount a host system file in my specific case?

What OS/distro are you running

$ cat /etc/os-release

NAME="Ubuntu" VERSION="16.04.4 LTS (Xenial Xerus)" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 16.04.4 LTS" VERSION_ID="16.04" HOME_URL="http://www.ubuntu.com/" SUPPORT_URL="http://help.ubuntu.com/" BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/" VERSION_CODENAME=xenial UBUNTU_CODENAME=xenial

How did you install Singularity

Our system admin installed that.

dtrudg commented 4 years ago

could not open alignment file result/2020_07_08_14_33_58/2020_07_08_14_33_58_1.bam is a relative path, so we can't be certain where the program is trying to open the file. The -B bind syntax you use should be fine.

What is the full actual command you run....

Instead of singularity run -B /path/to/data/folder:/data/ my_software.sif -i input1 [input2] (-r|-d) -o /data/ what are the real flags and paths? In case there is something being interpreted by the outer shell etc.

frankligy commented 4 years ago

Hi,

Thanks you for your kind reply, I got it work by running singularity run -B /path/to/data/folder:/mnt my_software.sif -i input1 [input2] (-r|-d) -o /mnt, I just change output folder to /mnt instead of /data. But it made me wonder the exact file system inside singularity, is there any way we could inspect the file system of a specific singularity container?

Thanks a lot, Frank

dtrudg commented 4 years ago

You can singularity exec mycontainer.sif ls / to get a view of the filesystem.

If it didn't work with /data then either the container is using /data already, or your system doesn't support mounting to non-existent locations (which needs overlay or underlay enabled).

I'd recommend also trying a newer version of Singularity if you can.

Glad you got it to work.