Private Docker Repo auth when using Fargate launch type

[sesh-kebab]

When using Fargate launch type, we aren't able to customize the underlying ec2 instances that the containers are run on. So our previous method adding docker auth via a Launch Configuration will no be possible.

Instead we will have to rely on ECR and credential helper already being on the instances that will pull the containers.

See discussion below: https://github.com/quintilesims/layer0/pull/525#discussion_r165734205

[sesh-kebab]

Apart from exposing the underlying ecr repository that is backing d.ims.io and recommending using the ecr-credential-helper on their dev and ci pipeline images, I can't think of what else can be done to ease the transition from d.ims.io to ecr repo.

Potential Workflow:

1. Create a new repo using d.ims.io
   • the response could return the Repository URI i.e. <accountid>.dkr.ecr.us-west-2.amazonaws.com/<repository name>
2. Use the URI returned in the response to tag images and push them directly to ECR
3. Fargate type Tasks would then be able to pull the images from ECR (as long as the ECR allows access from the AWS account the ECS services and tasks are running in).

As far as an actionable tasks as a result of this issue:

1. Update d.ims.io to expose underlying ecr repository URI
2. Update documentation to inform the user of the new workflow when using dynamic (default) environments

[sesh-kebab]

Alternative solution: ECR backed docker repository

Another potential solution could be a modification to L0 setup & our private-repo.ims.io which would allow the user to specify a docker image prefix mapping.

l0 setup: For example a config option like:

ECR_BACKED_REPO="private-repository.ims.io|12234567890.dkr.ecr.us-east-1.amazonaws.com"

Using this configuration value, when a user tries to create a deploy using an image like private-repository.io/seshi-kebab/guestbook, we would replace the private-repository.io part of the image name with 1223456789012.dkr.ecr.us-east-1.amazonaws.com.

This would allow a fargate launch type service/task to pull down the image and still keep the image private and have manageable names without the overly verbose ECR domain prefix. An additional benefit one could argue would be that private-repository.io going down would never prevent an image being pulled down from ECS.

private-repository.ims.io repository policies: For this to work, we would also require that each repository we are using in the mapping, allows cross account access in the cases where the service/task isn't in the same account as where we are hosting the ECR.

This would require a change to private-repository.io as well to create a policy like below for each new repository that is created via private-repository.ims.io and a way to update policies for all repositories every time we want to give/revoke access from a new aws account.

{
    "Version": "2008-10-17",
    "Statement": [
        {
            "Sid": "GrantCrossAccount",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::<otherAccountId>:*"
            },
            "Action": [
                "ecr:GetDownloadUrlForLayer",
                "ecr:BatchGetImage",
                "ecr:BatchCheckLayerAvailability"
            ]
        }
    ]
}

This solution on the surface might seem complicate given the simplicity of the user experience of private-repository.ims.io. But given the limitation of private docker repo auth with fargate, this change would introduce an extra step when creating a new instance, but would otherwise allow the fargate tasks to work with our private docker images in way that is transparent to the user.

https://docs.aws.amazon.com/AmazonECR/latest/userguide/set-repository-policy.html https://docs.aws.amazon.com/AmazonECR/latest/userguide/RepositoryPolicyExamples.html#IAM_allow_other_accounts