Force resolve dynamic secret references in change set creation

mschweitzer-sd commented 4 years ago

1. Title

2. Scope of request

Currently, if you have a dynamic reference to a secret, and don't specify the version, change set creation fails with the error message stating that there are no changes to the template, even if the secret value changed.

It would be great if we could pass either a flag or a capability to change set creation to force CloudFormation to detect and re-resolve dynamic references to secrets when you want the latest version of the secret.

We use a lot of CodePipeline, and for us, it is especially necessary for this to be supported by the CHANGE_SET_REPLACE CloudFormation action.

3. Expected behavior

When I leave off the version of a secret, it means I want to use the latest version, and that means when I change the secret value, I expect a change set to be created and for the change set to use the updated value.

4. Suggest specific test cases

Say you have a reference in a template Foo like so: {{resolve:secretsmanager:MySecret:SecretString:password}}

Go to SecretsManager
Retrieve and edit the password for MySecret
Create a change set for Foo (management console or CLI) --> Change set should be created and executed with the new password

Currently, change set creation fails.

5. Helpful Links to speed up research and evaluation

6. Category (required) - Will help with tagging and be easier to find by other users to +1

Security, Developer Tools, Enhancement

mliner commented 4 years ago

This issue is quite a blocker for me (I am using complex cloudformation setup) - same as referencing latest version of ssm parameter. This two improvements would really help hundreds of people working with complex cf setups.

@mschweitzer-sd I was thinking of a workaround - create a macro or custom resource which would update a cf output with time info in it - every create changeset will trigger the update as the time info won't be the same as previously set.

mschweitzer-sd commented 4 years ago

@mliner lol yeah I could see that working, neat idea!

dentonmwood commented 3 years ago

Please please please add this. CloudFormation doesn't have a great way to roll out changes which affect resources in other templates, and we would really appreciate having that functionality. We already had to give up native imports since errors are thrown when the changes cause new resources to be created, changing the values of the exported services.

dudeitssm commented 3 years ago

+1 for this. For now, I am ok with adding an extra input param (date), but a native solution would be much preferred.

mcIovin commented 1 year ago

Yes, please please please add this. We have some SAM stacks that use 'resolve' from secretsmanager in template.yaml, and without this functionality we cannot rotate the secret.

fabiatz commented 1 year ago

+1. If this is working with SSM parameters, it would be consistent to implement with SecretsManager entries, too.

yongzhang commented 1 year ago

Also blocked by this.

InQuirer commented 9 months ago

+1. our team ran into this problem too (e.g. if you want the API Gateway authorizer token to be copied up from the latest version of a secret)

  MyAuthorizer:
    Properties:
      IdentityValidationExpression: '{{resolve:secretsmanager:my-authorizer:SecretString:token}}'

yongzhang commented 5 months ago

Has anyone tried this? https://www.amazonaws.cn/en/new/2024/amazon-cloudformation-improves-changesets-to-enable-safer-deployment-practices/ https://aws.amazon.com/about-aws/whats-new/2024/04/aws-cloudformation-changesets-enhanced-change-visibility-deployments/

I did a test today but no luck, but I do see cfn created changeset successfully by updating secret values, unfortunately cfn didn't update my resource (ECS task definition) with updated secret value even though I can see "Replacement: true" in changeset.

InQuirer commented 5 months ago

for now, I did the following workaround @mliner proposed:

AWSTemplateFormatVersion: 2010-09-09
Resources:

    SecretExpanderMacro:
        Type: AWS::CloudFormation::Macro
        Properties:
            Name: SecretExpanderMacro # use this macro name is Transform section
            Description: Replaces resolve:secretsmanager with the exact latest version
            FunctionName: !Ref SecretExpanderLambda

    SecretExpanderLambda:
        Type: AWS::Serverless::Function
        Properties:
            CodeUri: ./expand_secrets_macro.py # see lambda code gist link below
            MemorySize: 128
            Timeout: 60 # seconds
            Handler: expand_secrets_macro.lambda_handler
            Runtime: python3.11
            Architectures:
                - x86_64
            EventInvokeConfig:
                MaximumEventAgeInSeconds: 60
                MaximumRetryAttempts: 2
            Policies:
                - Statement:
                      - Effect: Allow
                        Action:
                            - logs:CreateLogGroup
                            - logs:CreateLogStream
                            - logs:PutLogEvents
                        Resource:
                            - !Sub arn:aws:logs:${AWS::Region}:${AWS::AccountId}:*
                      - Effect: Allow
                        Action:
                            - secretsmanager:GetSecretValue
                        Resource:
                            - !Sub arn:aws:secretsmanager:${AWS::Region}:${AWS::AccountId}:secret:*

Transform: AWS::Serverless-2016-10-31

usage:

AWSTemplateFormatVersion: 2010-09-09
# ...

Transform:
  - SecretExpanderMacro

lambda code: https://gist.github.com/InQuirer/a7c5be8004ab4b78744a992ef58facc1

aws-cloudformation / cloudformation-coverage-roadmap