Show how to run workflows having private Docker images in the Amazon cloud

[brucehoff]

This thread discusses a solution for Toil: https://gitter.im/bd2k-genomics-toil/Lobby?at=5c59f014975714406b9da072

Asha Rostamianfar @arostamianfar Feb 04 16:00
Does anyone know how to use a private docker registry in AWS using the CWL runner? (basically, the answer to https://github.com/DataBiosphere/toil/issues/2166#issuecomment-402880526 ? )
...
I ended up working around it by creating a custom Toil appliance image that includes the amazon-ecr-credential-helper and setting TOIL_APPLIANCE_SELF accordingly....for reference, the new docker image is:
FROM quay.io/ucsc_cgl/toil:latest
RUN sudo printf '#!/bin/sh\n%s\n%s\n' \
    'SECRET=$(docker run --rm -e METHOD=$1 -e REGISTRY=$(cat -) pottava/amazon-ecr-credential-helper)' \
    'echo $SECRET | grep Secret' > \
    /usr/local/bin/docker-credential-ecr-login && \
    sudo chmod +x /usr/local/bin/docker-credential-ecr-login && \
    sudo mkdir ~/.docker && \
    sudo echo '{ "credsStore": "ecr-login" }' > ~/.docker/config.json

Although, I think it would be helpful if Toil hosted such an image for AWS-specific use cases as I think it's a pretty common use case

[thomasyu888]

I'm all fine with this but then wouldn't the docker container used by toil need to be private as well? If someone could access this docker container, they could potentially access all docker containers that this user has access to.

[brucehoff]

@thomasyu888 When you say, "the docker container used by toil" do you instead mean "the docker container containing toil" (i.e. the image created by the Dockerfile, above)? If so, I'm honestly not sure what the Dockerfile above is doing. Is it placing plain text credentials into the image? If so, then I agree it's a security hole and may not be what we want to recommend.

[thomasyu888]

Thanks for the clarification. Yes that's what I mean.

I think they are storing credentials in the config.json which would live in the docker container. I'm also confused because wouldn't the solution be to log into the docker registry that you want to interact with on your host machine?

[brucehoff]

wouldn't the solution be to log into the docker registry that you want to interact with on your host machine?

Are you asking whether it's the docker daemon on the host machine that has to docker pull from the registry and hence has to have the registry credentials? If so, I believe the answer is 'yes'.

[thomasyu888]

Thanks again for the clarification. That is indeed what I mean.