Solving the problems of running the Docker daemon as su on shared machines

[eclake]

Should solve #17 but want to keep this as separate issue to document progress.

Two alternatives suggested: Shifter and Singularity -both options can be run on HPC and also do not allow root escalation.

Shifter:

• Uses docker containers directly

Singularity:

• Can convert docker containers to singularity containers (require Docker and sudo privileges to make singularity/export docker containers).

Keeping docker images as primary distribution seems favourable given docker hub and ways to automatically build docker images from gitHub builds.

Would like to provide option to users for using Docker images on machines on which they do not have root access without them having to install many different programs.

Aiming for user themselves to provide conversion if conversion from docker images is required.

Singularity will require a linux machine with root access or a virtual machine if user has to build singularity container from our docker container therefore investigating Shifter first!

[eclake]

Update:

Unless I've missed the information somewhere, it seems as though shifter can only be used on RHEL/Centos/Scientific Linux 6 Systems.

I've been investigating Singularity instead and have managed to download it within an hour on two different operating systems: Ubuntu 14.04 via a virtualbox machine On my mac using Vagrant following their instructions here

Both of these options could be used to convert the container.

More to come once I know how easy it is to use said container, but hopefully this is a viable solution. Most people have access to a machine on which they have root privileges or on which they can run a virtual machine to produce the singularity container if they then want to run BEAGLE on a shared machine.

[jacopo-chevallard]

To run Beagle in Docker without being root user we must use the --user $(id -u):$(id -g) flag (note that you must use a bash shell), as this allows to create files with the right permission, solving the issues in #17.

I'm updating the wiki as well to reflect the update

[jacopo-chevallard]

Be careful that is Docker already created some folders owned by root, these folders cannot be re-written by a new user, and so they must be removed and re-created with the right ownership

[senchp]

Hey all - curious if anyone has successfully run BEAGLE with Singularity yet? I gave it a first go recently and ran into some strange error messages, but I don't have enough familiarity with Singularity at the moment to say whether they're due to BEAGLE peculiarities or just a mistake on my end.

[jacopo-chevallard]

Hey peter, I actually found the workaround above to run Beagle on Linux without root permission, perhaps no need for singularity anymore ! @senchp If you try the workaround, let us know if it works !

[senchp]

Thanks @jacopo-chevallard - unfortunately our HPC system doesn't have Docker installed at all, so Singularity is the only real option - will update if we manage to get it running!

[jacopo-chevallard]

ok, otherwise there is always the option of compiling it on the machine, but for this the best would be to let @eclake or I doing it...

[lukasfurtak]

Hi everyone! Is someone still trying to run BEAGLE with Singularity or even has succeeded? I have some problems translating the docker syntax to Singularity (our HPC cluster does not have docker and I don't have root privileges).

I first create a Singularity image .sif from the BEAGLE docker image with singularity pull docker://beagletool/beagle:0.23.0, which creates a file beagle_0.23.0.sif.

I then want to run this with singularity run so I basically tried to translate from the docker run command in the wiki: I suppose that --parameter-file and --fit/--mock are BEAGLE-internal and therefore won't change and I found out that the -v /Path/to/your/BEAGLE_workdir:/BEAGLE_virtual_workdir in docker would be -B /Path/to/your/BEAGLE_workdir:/BEAGLE_virtual_workdir in Singularity.

However, does anyone know how to translate the docker options --rm and -it to Singularity? And how to read in the list of environment variables?

[senchp]

A bit late on posting this - but I have successfully run BEAGLE now with Singularity in an HPC environment! Resolving these last few issues with the path definitions is fairly straightforward:

1. First, make a Singularity spec file that we'll build the image from. Here is where you can specify the BEAGLE version to pull from Docker Hub, and where you'll define the environment variables relative to the virtual work directory that BEAGLE tries to look for. I've appended a working example of such a file here.
2. Build the Singularity image from this spec file; this can be done on a linux or mac box that you have root access to (though on mac, you'll need to be logged in to Sylabs Cloud to conduct a remote build). The command on mac would then be singularity build --remote my_singularity_image.simg my_singularity_specfile.spec (on linux you should be able to omit --remote if you use sudo).
3. Then, you can should be able to run the resulting image on any HPC machine with a compatible version of Singularity installed and loaded into your path: e.g. singularity run -B /realpath/to/my/beagle_workdir/:/BEAGLE_virtual_workdir my_beagle_singularity_image.simg --parameter-file /BEAGLE_virtual_workdir/params/my_param_file.param --fit.

Let me know if anyone tries this and runs into trouble in their environment! Otherwise, we should be able to close this issue I think, unless I should first add this to the docs somewhere?

Bootstrap: docker
From: beagletool/beagle:0.19.9

%environment
    BEAGLE_TEMPLATES=/BEAGLE_virtual_workdir/templates
    BEAGLE_FILTERS=/BEAGLE_virtual_workdir/filters
    BEAGLE_DATA=/BEAGLE_virtual_workdir/data
    BEAGLE_SF_CHE=/BEAGLE_virtual_workdir/sf_che
    BEAGLE_RESULTS=/BEAGLE_virtual_workdir/results
    BEAGLE_DUST=/BEAGLE_virtual_workdir/dust
    BEAGLE_PARAM_DIR=/BEAGLE_virtual_workdir/params
    FILTERS=/BEAGLE_virtual_workdir/filters/FILTERBIN.RES
    export BEAGLE_TEMPLATES
    export BEAGLE_FILTERS
    export BEAGLE_DATA
    export BEAGLE_SF_CHE
    export BEAGLE_RESULTS
    export BEAGLE_DUST
    export BEAGLE_PARAM_DIR
    export FILTERS

[lukasfurtak]

Hi Peter, thanks for your post, I'd say it's never to late! Just one question: How compulsory is it to have root access? I don't have that on our cluster...

[senchp]

I think you need root access to build the image from a spec file (not 100% sure); but you should be able to build it locally then transfer it to the cluster where you can then run it as a normal user!

[lukasfurtak]

Ok, after testing this also works for me. Note that the singularity build --remote did NOT work on my Mac but sudo singularity build did for some reason. I also think that this should be added to the documentation or wiki somewhere, this can be very useful I think.

[senchp]

Awesome, thanks for testing @lukasfurtak!! I've added these instructions to the wiki now; until someone runs into trouble using those instructions, I think we can safely close this issue.