awslabs / ecs-refarch-continuous-deployment

ECS Reference Architecture for creating a flexible and scalable deployment pipeline to Amazon ECS using AWS CodePipeline
https://aws.amazon.com/blogs/compute/continuous-deployment-to-amazon-ecs-using-aws-codepipeline-aws-codebuild-amazon-ecr-and-aws-cloudformation/
Apache License 2.0
850 stars 999 forks source link

Image tagging method cannot be used with container repository lifecycle policies #38

Open jinty opened 6 years ago

jinty commented 6 years ago

I am trying to setup a stack using you method of tagging images but adding in a container lifecycle policy on the repository to delete old images. Most images get tags so in order to delete them I am required to add a "tagPrefixList". adding a tag prefix list ["0", "1", "2", "3", "4", "5", "6", "7", "8", "9", "a", "b", "c", "d", "e", "f"] causes an "Internal Server Error" and timeout in cloudformation...

Here's a excerpt from my .yaml file:

ContainerRepository:
    Type: "AWS::ECR::Repository"
    Properties:
      RepositoryName: test-repository
      LifecyclePolicy:
        LifecyclePolicyText: |
          {
              "rules": [
                  {
                      "rulePriority": 1,
                      "description": "Keep only 20 untagged image, expire all others",
                      "selection": {
                          "tagStatus": "untagged",
                          "countType": "imageCountMoreThan",
                          "countNumber": 20
                      },
                      "action": {
                          "type": "expire"
                      }
                  },
                  {
                      "rulePriority": 2,
                      "description": "Keep only 20 tagged image, expire all others",
                      "selection": {
                          "tagStatus": "tagged",
                          "tagPrefixList": ["0", "1", "2", "3", "4", "5", "6", "7", "8", "9", "a", "b", "c", "d", "e", "f"],
                          "countType": "imageCountMoreThan",
                          "countNumber": 20
                      },
                      "action": {
                          "type": "expire"
                      }
                  }
              ]
          }
jpignata commented 6 years ago

Hmm - interesting - what if you prefixed the tags in the build specification with prod_ or somesuch and provided that as the tag prefix in the list?

SunlightJoe commented 6 years ago

If you specify multiple tags, only images with all specified tags are selected.

https://docs.aws.amazon.com/AmazonECR/latest/userguide/LifecyclePolicies.html#lp_tag_prefix_list

@jinty I think you're trying to do the opposite of what that does.

jinty commented 6 years ago

On Fri, Mar 30, 2018 at 05:00:38PM +0000, John Pignata wrote:

Hmm - interesting - what if you prefixed the tags in the build specification with prod_ or somesuch and provided that as the tag prefix in the list?

@jpignata Yeah, that works! I added the prefix "commit-" which is slightly more generic.

With such an easy workaround, I'll just close the issue. Unless you want to add cleanup to the refarch?

-- Brian Sutherland

jinty commented 6 years ago

On Fri, Mar 30, 2018 at 06:27:30PM -0700, Joe Hillenbrand wrote:

If you specify multiple tags, only images with all specified tags are selected.

https://docs.aws.amazon.com/AmazonECR/latest/userguide/LifecyclePolicies.html#lp_tag_prefix_list

@jinty I think you're trying to do the opposite of what that does.

Yeah, I didn't read the docs thoroughly. Even so, it does not appear possible to make a generic cleanup rule that will cleanup all images regardless of their tags.

-- Brian Sutherland

jpignata commented 6 years ago

You should be able to do this without a prefix. I think the field is required, but can be empty. I haven't had a chance to test this yet. The walkthrough in the docs explicitly specifies that the prefix list is optional.

SunlightJoe commented 6 years ago

When I push this policy:

      LifecyclePolicy:
        LifecyclePolicyText: |
            {
              "rules": [
                {
                  "rulePriority": 1,
                  "description": "Only keep untagged images for 7 days",
                  "selection": {
                    "tagStatus": "untagged",
                    "countType": "sinceImagePushed",
                    "countUnit": "days",
                    "countNumber": 7
                  },
                  "action": { "type": "expire" }
                },
                {
                  "rulePriority": 2,
                  "description": "Keep only 10 tagged images, expire all others",
                  "selection": {
                    "tagStatus": "tagged",
                    "countType": "imageCountMoreThan",
                    "countNumber": 10
                  },
                  "action": { "type": "expire" }
                }
              ]
            }

I get this error:

Invalid parameter at 'LifecyclePolicyText' failed to satisfy constraint: 'Lifecycle policy valiation failure: Must specify tagPrefixList when tagStatus=TAGGED.'

I also get different errors if I try "tagPrefixList": [], or "tagPrefixList": [""],

I've just put a g in front of all my tags and set "tagPrefixList": ["g"],. It's a hack but it's all I've got.