search

aws / aws-cdk

The AWS Cloud Development Kit is a framework for defining cloud infrastructure in code
https://aws.amazon.com/cdk
Apache License 2.0
11.5k stars 3.84k forks source link

(secrets): How do we force Secret resolution in a particular scope? #28845

Open diranged opened 7 months ago

diranged commented 7 months ago

Describe the bug

Ref: https://cdk-dev.slack.com/archives/C018XT6REKT/p1706125503162629

I want to resolve a secret ({{resolve:secretsmanager...}} within the context of a parent Stack, and pass the parameter through to a NestedStack. This will allow the nested stack to use the parameter in Custom::xxx resources. This is a pattern we have in many of our legacy YAML based CloudFOrmation stacks and it works well.

When I try to do this, CDK seems to do the resolution in the nested stack itself.. this resolution then fails, and the custom resource will not work. It seems odd to me that a Secret would be resolved (turned into {{resolve:...}} in ANY context other than the scope passed to it.

Expected Behavior

I expect the resolution to occur in the parent stack where I have passed the scope in.

Current Behavior

It is resolved in the nested stack.

Reproduction Steps

# stack.ts
/** @format */

import { CustomResource, NestedStack, NestedStackProps, Stack } from 'aws-cdk-lib';
import { Construct } from 'constructs';
import { Secret } from 'aws-cdk-lib/aws-secretsmanager';

export interface AppStackProps extends NestedStackProps {
  readonly apikey: string;
}

export class ExampleStack extends Stack {
  constructor(scope: Construct, id: string) {
    super(scope, id);
    new AppStack(this, 'appStack', {
      apikey: Secret.fromSecretNameV2(this, 'ApiKey', '/secret').secretValue.toString(),
    });
  }
}

export class AppStack extends NestedStack {
  constructor(scope: Construct, id: string, props: AppStackProps) {
    super(scope, id);

    new CustomResource(this, 'Custom::thingy', {
      serviceToken: 'arn:to:lambda',
      properties: {
        token: props.apikey,
      },
    });
  }
}

Generate it...

yarn cdk synth

Check out the output:

# template.json
{
 "Resources": {
  "appStackNestedStackappStackNestedStackResourceA44C89FF": {
   "Type": "AWS::CloudFormation::Stack",
   "Properties": {
    "TemplateURL": {
     "Fn::Join": [
      "",
      [
       "https://s3.us-west-2.",
       {
        "Ref": "AWS::URLSuffix"
       },
       "/cdk-hnb659fds-assets-XXXXX-us-west-2/aa703b1fd367d9aecc9683745cac0b97d90123b2d2a96a4014499c95009bf223.json"
      ]
     ]
    }
   },
   "UpdateReplacePolicy": "Delete",
   "DeletionPolicy": "Delete",
   "Metadata": {
...
   }
  },
  "CDKMetadata": {
   "Type": "AWS::CDK::Metadata",
   "Properties": {   },
   "Metadata": {
    "aws:cdk:path": "Stage/ExampleStack/CDKMetadata/Default"
   }
  }
 },
 "Parameters": {
  "BootstrapVersion": {
   "Type": "AWS::SSM::Parameter::Value<String>",
   "Default": "/cdk-bootstrap/hnb659fds/version",
   "Description": "Version of the CDK Bootstrap resources in this environment, automatically retrieved from SSM Parameter Store. [cdk:skip]"
  }
 },
 "Rules": {
  "CheckBootstrapVersion": {
   "Assertions": [
    {
     "Assert": {
      "Fn::Not": [
       {
        "Fn::Contains": [
         [
          "1",
          "2",
          "3",
          "4",
          "5"
         ],
         {
          "Ref": "BootstrapVersion"
         }
        ]
       }
      ]
     },
     "AssertDescription": "CDK bootstrap stack version 6 required. Please run 'cdk bootstrap' with a recent version of the CDK CLI."
    }
   ]
  }
 }
}
# nested.template.json
{
 "Resources": {
  "Customthingy": {
   "Type": "AWS::CloudFormation::CustomResource",
   "Properties": {
    "ServiceToken": "arn:to:lambda",
    "token": {
     "Fn::Join": [
      "",
      [
       "{{resolve:secretsmanager:arn:",
       {
        "Ref": "AWS::Partition"
       },
       ":secretsmanager:us-west-2:XXXXX:secret:/secret:SecretString:::}}"
      ]
     ]
    }
   },
   "UpdateReplacePolicy": "Delete",
   "DeletionPolicy": "Delete",
   "Metadata": {
    "aws:cdk:path": "Stage/ExampleStack/appStack/Custom::thingy/Default"
   }
  },
  "CDKMetadata": {
   "Type": "AWS::CDK::Metadata",
   "Properties": {   },
   "Metadata": {
    "aws:cdk:path": "Stage/ExampleStack/appStack/CDKMetadata/Default"
   }
  }
 }

Possible Solution

No response

Additional Information/Context

No response

CDK CLI Version

2.115.0

Framework Version

No response

Node.js Version

18

OS

osx

Language

TypeScript

Language Version

No response

Other information

No response

tim-finnigan commented 7 months ago

Hi @diranged thanks for reaching out. Here is documentation on Secret and SecretValue for reference.

Have you tried using fromSecretAttributes as shown in this example: https://docs.aws.amazon.com/cdk/v2/guide/get_secrets_manager_value.html ? Does the secret resolve upon deploying or what error do you get when trying this?

github-actions[bot] commented 7 months ago

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

diranged commented 7 months ago

@tim-finnigan, Is this what you mean? If so, it doesn't help because the secret resolution is still happening in the nested stack...

Test 1:

# example.ts
/** @format */

import { CustomResource, NestedStack, NestedStackProps, Stack } from 'aws-cdk-lib';
import { Construct } from 'constructs';
import { Secret } from 'aws-cdk-lib/aws-secretsmanager';

export interface AppStackProps extends NestedStackProps {
  readonly apikeyarn: string;
}

export class ExampleStack extends Stack {
  constructor(scope: Construct, id: string) {
    super(scope, id);
    new AppStack(this, 'appStack', {
      apikeyarn: Secret.fromSecretNameV2(this, 'ApiKey', '/secret').secretArn,
    });
  }
}

export class AppStack extends NestedStack {
  constructor(scope: Construct, id: string, props: AppStackProps) {
    super(scope, id);

    const secret = Secret.fromSecretAttributes(this, 'ImportedSecret', {
      secretCompleteArn: props.apikeyarn,
    });

    new CustomResource(this, 'Custom::thingy', {
      serviceToken: 'arn:to:lambda',
      properties: {
        token: secret.secretValue.toString(),
      },
    });
  }
}
% yarn cdk synth ExampleStack
...
Resources:
  appStackNestedStackappStackNestedStackResourceA44C89FF:
    Type: AWS::CloudFormation::Stack
    Properties:
      TemplateURL:
        Fn::Join:
          - ""
          - - https://s3.
            - Ref: AWS::Region
            - "."
            - Ref: AWS::URLSuffix
            - /
            - Fn::Sub: cdk-hnb659fds-assets-${AWS::AccountId}-${AWS::Region}
            - /65067764932937f66cc12eaca3ea465cd12c9a867b2f9dc48468bb6167dc2a2f.json
    UpdateReplacePolicy: Delete
    DeletionPolicy: Delete
    Metadata:
      aws:cdk:path: ExampleStack/appStack.NestedStack/appStack.NestedStackResource
      aws:asset:path: ExampleStackappStackD9CE9E37.nested.template.json
      aws:asset:property: TemplateURL
  CDKMetadata:
    Type: AWS::CDK::Metadata
    Properties:
      Analytics: v2:deflate64:H4sIAAAAAAAA/zPSMzQ01zNQTCwv1k1OydbNyUzSqw4uSUzO1glKLc4vLUpO1XFOy4OIABnIgs75eSmZJZn5ebU6efkpqXpZxfplhhZ6hpZA87KKMzN1i0rzSjJzU/WCIDQAC7W+QWwAAAA=
    Metadata:
      aws:cdk:path: ExampleStack/CDKMetadata/Default
    Condition: CDKMetadataAvailable
Conditions:
  CDKMetadataAvailable:
    Fn::Or:
      - Fn::Or:
          - Fn::Equals:
              - Ref: AWS::Region
              - af-south-1
          - Fn::Equals:
              - Ref: AWS::Region
              - ap-east-1
          - Fn::Equals:
              - Ref: AWS::Region
              - ap-northeast-1
          - Fn::Equals:
              - Ref: AWS::Region
              - ap-northeast-2
          - Fn::Equals:
              - Ref: AWS::Region
              - ap-south-1
          - Fn::Equals:
              - Ref: AWS::Region
              - ap-southeast-1
          - Fn::Equals:
              - Ref: AWS::Region
              - ap-southeast-2
          - Fn::Equals:
              - Ref: AWS::Region
              - ca-central-1
          - Fn::Equals:
              - Ref: AWS::Region
              - cn-north-1
          - Fn::Equals:
              - Ref: AWS::Region
              - cn-northwest-1
      - Fn::Or:
          - Fn::Equals:
              - Ref: AWS::Region
              - eu-central-1
          - Fn::Equals:
              - Ref: AWS::Region
              - eu-north-1
          - Fn::Equals:
              - Ref: AWS::Region
              - eu-south-1
          - Fn::Equals:
              - Ref: AWS::Region
              - eu-west-1
          - Fn::Equals:
              - Ref: AWS::Region
              - eu-west-2
          - Fn::Equals:
              - Ref: AWS::Region
              - eu-west-3
          - Fn::Equals:
              - Ref: AWS::Region
              - il-central-1
          - Fn::Equals:
              - Ref: AWS::Region
              - me-central-1
          - Fn::Equals:
              - Ref: AWS::Region
              - me-south-1
          - Fn::Equals:
              - Ref: AWS::Region
              - sa-east-1
      - Fn::Or:
          - Fn::Equals:
              - Ref: AWS::Region
              - us-east-1
          - Fn::Equals:
              - Ref: AWS::Region
              - us-east-2
          - Fn::Equals:
              - Ref: AWS::Region
              - us-west-1
          - Fn::Equals:
              - Ref: AWS::Region
              - us-west-2
Parameters:
  BootstrapVersion:
    Type: AWS::SSM::Parameter::Value<String>
    Default: /cdk-bootstrap/hnb659fds/version
    Description: Version of the CDK Bootstrap resources in this environment, automatically retrieved from SSM Parameter Store. [cdk:skip]
Rules:
  CheckBootstrapVersion:
    Assertions:
      - Assert:
          Fn::Not:
            - Fn::Contains:
                - - "1"
                  - "2"
                  - "3"
                  - "4"
                  - "5"
                - Ref: BootstrapVersion
        AssertDescription: CDK bootstrap stack version 6 required. Please run 'cdk bootstrap' with a recent version of the CDK CLI.

✨  Done in 8.88s.
{
 "Resources": {
  "Customthingy": {
   "Type": "AWS::CloudFormation::CustomResource",
   "Properties": {
    "ServiceToken": "arn:to:lambda",
    "token": {
     "Fn::Join": [
      "",
      [
       "{{resolve:secretsmanager:arn:",
       {
        "Ref": "AWS::Partition"
       },
       ":secretsmanager:",
       {
        "Ref": "AWS::Region"
       },
       ":",
       {
        "Ref": "AWS::AccountId"
       },
       ":secret:/secret:SecretString:::}}"
      ]
     ]
    }
   },
   "UpdateReplacePolicy": "Delete",
   "DeletionPolicy": "Delete",
   "Metadata": {
    "aws:cdk:path": "ExampleStack/appStack/Custom::thingy/Default"
   }
  },
  "CDKMetadata": {
   "Type": "AWS::CDK::Metadata",
   "Properties": {
    "Analytics": "v2:deflate64:H4sIAAAAAAAA/zPSMzQ01zNQTCwv1k1OydbNyUzSq/ZLLS5JTQkuSUzO1glKLc4vLUpO1XEuLS7Jz0Vw0/KQ2c75eSmZJZn5ebU6efkpqXpZxfplhhZ6hpZAs7OKMzN1i0rzSjJzU/WCIDQAjYGvWXgAAAA="
   },
   "Metadata": {
    "aws:cdk:path": "ExampleStack/appStack/CDKMetadata/Default"
   },
   "Condition": "CDKMetadataAvailable"
  }
 },
 "Conditions": {
  "CDKMetadataAvailable": {
   "Fn::Or": [
    {
     "Fn::Or": [
      {
       "Fn::Equals": [
        {
         "Ref": "AWS::Region"
        },
        "af-south-1"
       ]
      },
      {
       "Fn::Equals": [
        {
         "Ref": "AWS::Region"
        },
        "ap-east-1"
       ]
      },
      {
       "Fn::Equals": [
        {
         "Ref": "AWS::Region"
        },
        "ap-northeast-1"
       ]
      },
      {
       "Fn::Equals": [
        {
         "Ref": "AWS::Region"
        },
        "ap-northeast-2"
       ]
      },
      {
       "Fn::Equals": [
        {
         "Ref": "AWS::Region"
        },
        "ap-south-1"
       ]
      },
      {
       "Fn::Equals": [
        {
         "Ref": "AWS::Region"
        },
        "ap-southeast-1"
       ]
      },
      {
       "Fn::Equals": [
        {
         "Ref": "AWS::Region"
        },
        "ap-southeast-2"
       ]
      },
      {
       "Fn::Equals": [
        {
         "Ref": "AWS::Region"
        },
        "ca-central-1"
       ]
      },
      {
       "Fn::Equals": [
        {
         "Ref": "AWS::Region"
        },
        "cn-north-1"
       ]
      },
      {
       "Fn::Equals": [
        {
         "Ref": "AWS::Region"
        },
        "cn-northwest-1"
       ]
      }
     ]
    },
    {
     "Fn::Or": [
      {
       "Fn::Equals": [
        {
         "Ref": "AWS::Region"
        },
        "eu-central-1"
       ]
      },
      {
       "Fn::Equals": [
        {
         "Ref": "AWS::Region"
        },
        "eu-north-1"
       ]
      },
      {
       "Fn::Equals": [
        {
         "Ref": "AWS::Region"
        },
        "eu-south-1"
       ]
      },
      {
       "Fn::Equals": [
        {
         "Ref": "AWS::Region"
        },
        "eu-west-1"
       ]
      },
      {
       "Fn::Equals": [
        {
         "Ref": "AWS::Region"
        },
        "eu-west-2"
       ]
      },
      {
       "Fn::Equals": [
        {
         "Ref": "AWS::Region"
        },
        "eu-west-3"
       ]
      },
      {
       "Fn::Equals": [
        {
         "Ref": "AWS::Region"
        },
        "il-central-1"
       ]
      },
      {
       "Fn::Equals": [
        {
         "Ref": "AWS::Region"
        },
        "me-central-1"
       ]
      },
      {
       "Fn::Equals": [
        {
         "Ref": "AWS::Region"
        },
        "me-south-1"
       ]
      },
      {
       "Fn::Equals": [
        {
         "Ref": "AWS::Region"
        },
        "sa-east-1"
       ]
      }
     ]
    },
    {
     "Fn::Or": [
      {
       "Fn::Equals": [
        {
         "Ref": "AWS::Region"
        },
        "us-east-1"
       ]
      },
      {
       "Fn::Equals": [
        {
         "Ref": "AWS::Region"
        },
        "us-east-2"
       ]
      },
      {
       "Fn::Equals": [
        {
         "Ref": "AWS::Region"
        },
        "us-west-1"
       ]
      },
      {
       "Fn::Equals": [
        {
         "Ref": "AWS::Region"
        },
        "us-west-2"
       ]
      }
     ]
    }
   ]
  }
 }
}

Test 2

/** @format */

import { CustomResource, NestedStack, NestedStackProps, Stack } from 'aws-cdk-lib';
import { Construct } from 'constructs';
import { Secret } from 'aws-cdk-lib/aws-secretsmanager';

export interface AppStackProps extends NestedStackProps {
  readonly apikey: string;
}

export class ExampleStack extends Stack {
  constructor(scope: Construct, id: string) {
    super(scope, id);
    new AppStack(this, 'appStack', {
      apikey: Secret.fromSecretAttributes(this, 'ImportedSecret', {
        secretCompleteArn: 'arn:aws:secretsmanager:us-east-1:11111111111:secret:/mysecret-abcdef',
      }).secretValue.toString(),
    });
  }
}

export class AppStack extends NestedStack {
  constructor(scope: Construct, id: string, props: AppStackProps) {
    super(scope, id);

    new CustomResource(this, 'Custom::thingy', {
      serviceToken: 'arn:to:lambda',
      properties: {
        token: props.apikey,
      },
    });
  }
}

synth output:

Resources:
  appStackNestedStackappStackNestedStackResourceA44C89FF:
    Type: AWS::CloudFormation::Stack
    Properties:
      TemplateURL:
        Fn::Join:
          - ""
          - - https://s3.
            - Ref: AWS::Region
            - "."
            - Ref: AWS::URLSuffix
            - /
            - Fn::Sub: cdk-hnb659fds-assets-${AWS::AccountId}-${AWS::Region}
            - /2d0b13de0dfd23792951357b1586093841b3c14dc8f042c2c84f548bf870506e.json
    UpdateReplacePolicy: Delete
    DeletionPolicy: Delete
    Metadata:
      aws:cdk:path: ExampleStack/appStack.NestedStack/appStack.NestedStackResource
      aws:asset:path: ExampleStackappStackD9CE9E37.nested.template.json
      aws:asset:property: TemplateURL
  CDKMetadata:
    Type: AWS::CDK::Metadata
    Properties:
      Analytics: v2:deflate64:H4sIAAAAAAAA/zPSMzQ01zNQTCwv1k1OydbNyUzSqw4uSUzO1glKLc4vLUpO1XFOy4OIABnIgs75eSmZJZn5ebU6efkpqXpZxfplhhZ6hpZA87KKMzN1i0rzSjJzU/WCIDQAC7W+QWwAAAA=
    Metadata:
      aws:cdk:path: ExampleStack/CDKMetadata/Default
    Condition: CDKMetadataAvailable
Conditions:
  CDKMetadataAvailable:
    Fn::Or:
      - Fn::Or:
          - Fn::Equals:
              - Ref: AWS::Region
              - af-south-1
          - Fn::Equals:
              - Ref: AWS::Region
              - ap-east-1
          - Fn::Equals:
              - Ref: AWS::Region
              - ap-northeast-1
          - Fn::Equals:
              - Ref: AWS::Region
              - ap-northeast-2
          - Fn::Equals:
              - Ref: AWS::Region
              - ap-south-1
          - Fn::Equals:
              - Ref: AWS::Region
              - ap-southeast-1
          - Fn::Equals:
              - Ref: AWS::Region
              - ap-southeast-2
          - Fn::Equals:
              - Ref: AWS::Region
              - ca-central-1
          - Fn::Equals:
              - Ref: AWS::Region
              - cn-north-1
          - Fn::Equals:
              - Ref: AWS::Region
              - cn-northwest-1
      - Fn::Or:
          - Fn::Equals:
              - Ref: AWS::Region
              - eu-central-1
          - Fn::Equals:
              - Ref: AWS::Region
              - eu-north-1
          - Fn::Equals:
              - Ref: AWS::Region
              - eu-south-1
          - Fn::Equals:
              - Ref: AWS::Region
              - eu-west-1
          - Fn::Equals:
              - Ref: AWS::Region
              - eu-west-2
          - Fn::Equals:
              - Ref: AWS::Region
              - eu-west-3
          - Fn::Equals:
              - Ref: AWS::Region
              - il-central-1
          - Fn::Equals:
              - Ref: AWS::Region
              - me-central-1
          - Fn::Equals:
              - Ref: AWS::Region
              - me-south-1
          - Fn::Equals:
              - Ref: AWS::Region
              - sa-east-1
      - Fn::Or:
          - Fn::Equals:
              - Ref: AWS::Region
              - us-east-1
          - Fn::Equals:
              - Ref: AWS::Region
              - us-east-2
          - Fn::Equals:
              - Ref: AWS::Region
              - us-west-1
          - Fn::Equals:
              - Ref: AWS::Region
              - us-west-2
Parameters:
  BootstrapVersion:
    Type: AWS::SSM::Parameter::Value<String>
    Default: /cdk-bootstrap/hnb659fds/version
    Description: Version of the CDK Bootstrap resources in this environment, automatically retrieved from SSM Parameter Store. [cdk:skip]
Rules:
  CheckBootstrapVersion:
    Assertions:
      - Assert:
          Fn::Not:
            - Fn::Contains:
                - - "1"
                  - "2"
                  - "3"
                  - "4"
                  - "5"
                - Ref: BootstrapVersion
        AssertDescription: CDK bootstrap stack version 6 required. Please run 'cdk bootstrap' with a recent version of the CDK CLI.

% more cdk.out/ExampleStackappStackD9CE9E37.nested.template.json
{
 "Resources": {
  "Customthingy": {
   "Type": "AWS::CloudFormation::CustomResource",
   "Properties": {
    "ServiceToken": "arn:to:lambda",
    "token": "{{resolve:secretsmanager:arn:aws:secretsmanager:us-east-1:11111111111:secret:/mysecret-abcdef:SecretString:::}}"
   },
   "UpdateReplacePolicy": "Delete",
   "DeletionPolicy": "Delete",
   "Metadata": {
    "aws:cdk:path": "ExampleStack/appStack/Custom::thingy/Default"
   }
  },
  "CDKMetadata": {
   "Type": "AWS::CDK::Metadata",
   "Properties": {
    "Analytics": "v2:deflate64:H4sIAAAAAAAA/zPSMzQ01zNQTCwv1k1OydbNyUzSq/ZLLS5JTQkuSUzO1nEuLS7Jzw1KLc4vLUpO1XFOy0NmO+fnpWSWZObn1erk5aek6mUV65cZWugZWgKNzCrOzNQtKs0rycxN1QuC0ACzXQhFbwAAAA=="
   },
   "Metadata": {
    "aws:cdk:path": "ExampleStack/appStack/CDKMetadata/Default"
   },
   "Condition": "CDKMetadataAvailable"
  }
 },
...