(aws-ecs): Grant permission to decrypt secret encrypted using CMK

[kierans]

What is the problem?

When using a secret held in AWS Secrets Manager that is encrypted using a CMK, when deploying an Fargate Task Definition the Task Execution Role policy is not updated to allow using the the key to decrypt the secret.

The consequence is that the task fails to deploy with a resource initialisation error.

Reproduction Steps

1. Create a key/secret in a stack

class SecurityStack extends cdk.Stack {
  constructor(scope: cdk.Construct, id: string, props: cdk.StackProps) {
    super(scope, id, props);

    const key = new kms.Key(this, "SecretsKey", {
      alias: `myapp/secrets`,
    });

    const secret = new secrets.Secret(this, "AppSecret", {
      secretName: "prod/app/mysecret",
      encryptionKey: key
    });
  }
}

2. Create a Fargate task def that uses the key/secret

class AppStack extends cdk.Stack {
  constructor(scope: cdk.Construct, id: string, props: cdk.StackProps) {
    super(scope, id, props);

    const secretsKey = kms.Key.fromLookup(this, "SecretsKey", {
      aliasName: "alias/myapp/secrets"
    });

    // Do other init for task def ...

    taskDef.addContainer("Container", {
      // ...

      secrets: {
        SECRET_VALUE: ecs.Secret.fromSecretsManager(
          secrets.Secret.fromSecretAttributes(
            this,
            "AppSecret",
            {
              secretPartialArn: "arn:aws:secretsmanager:ap-southeast-2:123456789:secret:prod/app/mysecret",
              encryptionKey: secretsKey
            }
          ),
          "secretValue"
        )
      }

      // ...
    });   
  }
}

What did you expect to happen?

That the Fargate task would deploy and the secret would be injected into the container.

What actually happened?

The container creation fails with the error

ResourceInitializationError: unable to pull secrets or registry auth: execution resource retrieval failed: unable to retrieve secret from asm: service call has been retried 1 time(s): failed to fetch secret arn:aws:secretsmanager (...)

The error message is unhelpful which means it took a number of hours to track down the source of the issue (cf: https://github.com/aws/containers-roadmap/issues/1133)

If you look at the CF, the Execution Role policy is missing the KMS permissions (cf: https://docs.aws.amazon.com/AmazonECS/latest/userguide/specifying-sensitive-data-secrets.html#secrets-iam)

CDK CLI Version

1.123.0

Framework Version

1.123.0

Node.js Version

v12.22.2

OS

macOS 10.15.7

Language

Typescript

Language Version

3.9.10

Other information

I managed to overcome the issue by manually adding a PolicyStatement

taskDef.addToExecutionRolePolicy(new iam.PolicyStatement({
  effect: iam.Effect.ALLOW,
  actions: [
    "kms:Decrypt"
  ],
  resources: [
    secretsKey.keyArn
  ]
}));

I'm not sure, but I wonder if this issue is related to https://github.com/aws/aws-cdk/issues/10160

[madeline-k]

Thanks for opening this issue and providing a workaround, @kierans! I think you are right, this does seem to be related to https://github.com/aws/aws-cdk/issues/10160.

[mannau]

I can confirm the issue. The proposed workaround did the trick!

Any updates on this?