(cdk-core): (Cfn Parameter value is not updated when source SSM Parameter is renamed)

nikovirtala commented 2 years ago

Describe the bug

When CDK creates CloudFormation Stack Parameter from SSM Parameter, and the SSM Parameter name is changed, there is no or no easy way to get the change deployed.

Expected Behavior

CloudFormation Stack Parameter value will be read from the renamed SSM Parameter.

Current Behavior

CloudFormation Stack Parameter value stuck to the initially deployed SSM Parameter.

Reproduction Steps

[ ] Create SSM Parameter "X" with e.g. AWS CLI: aws ssm put-parameter --name "X" --type String --value "something something"
[ ] Create SSM Parameter "Y" with e.g. AWS CLI: aws ssm put-parameter --name "Y" --type String --value "something new"
[ ] Deploy the following CDK app to the same account/region you created the SSM Parameters in the previous steps

import { App, Stack, aws_ssm } from 'aws-cdk-lib';
import { Construct } from 'constructs';

export class MyStack extends Stack {
  constructor(scope: Construct, id: string) {
    super(scope, id);

    /* create SSM parameter using aws cli:
     * aws ssm put-parameter --name "X" --type String --value "something something"
     */

    /* create another SSM parameter using aws cli:
     * aws ssm put-parameter --name "Y" --type String --value "something new"
     */

    /* delete the original SSM parameter:
     * aws ssm delete-parameter --name "X"
     */

    // toggle this parameter value between "X" and "Y" to see the difference
    const parameterName = 'X';

    // read already existing SSM parameter X
    const x = aws_ssm.StringParameter.fromStringParameterAttributes(this, 'X', {
      parameterName: parameterName,
    }).stringValue;

    // write parameter X value into new SSM parameter Y
    new aws_ssm.StringParameter(this, 'Z', {
      stringValue: x,
    });
  }
}

const app = new App();

new MyStack(app, 'UpdateParameters');
app.synth();

[ ] Toggle variable parameterName value to Y
[ ] Run cdk diff, it should show following:

Stack UpdateParameters
Parameters
[~] Parameter X.Parameter XParameter: {"Type":"AWS::SSM::Parameter::Value<String>","Default":"X"} to {"Type":"AWS::SSM::Parameter::Value<String>","Default":"Y"}

[ ] Deploy the updated stack – it shows "no changes" (which is unexpected)

UpdateParameters: deploying...
...
UpdateParameters: creating CloudFormation changeset...

 ✅  UpdateParameters (no changes)

[ ] Delete the SSM Parameter "X" with e.g. AWS CLI: aws ssm delete-parameter --name "X"
[ ] Deploy the updated stack – it fails!

UpdateParameters: deploying...
...
UpdateParameters: creating CloudFormation changeset...

 ❌  UpdateParameters failed: Error [ValidationError]: Unable to fetch parameters [X] from parameter store for this account.

Unable to fetch parameters [X] from parameter store for this account.

Possible Solution

Allow user to set "UsePreviousValue": false for Parameters when creating CloudFormation ChangeSet – that's how you solve this type of problem in CloudFormation.

Additional Information/Context

CDK verbose output when trying to create ChangeSet:

UpdateParameters: creating CloudFormation changeset...
Call failed: createChangeSet({
  "StackName": "UpdateParameters",
  "ChangeSetName": "cdk-deploy-change-set",
  "ChangeSetType": "UPDATE",
  "Description": "CDK Changeset for execution xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  "TemplateURL": "https://s3.eu-west-1.amazonaws.com/cdk-hnb659fds-assets-111111111111-eu-west-1/cefdc5a2b337755dcd2047e2dac56831b5a4155ebdebe6939e040f4d5315e1bc.json",
  "Parameters": [
    { "ParameterKey": "XParameter", "UsePreviousValue": true },
    { "ParameterKey": "BootstrapVersion", "UsePreviousValue": true }
  ],
  "RoleARN": "arn:aws:iam::111111111111:role/cdk-hnb659fds-cfn-exec-role-111111111111-eu-west-1",
  "Capabilities": [
    "CAPABILITY_IAM",
    "CAPABILITY_NAMED_IAM",
    "CAPABILITY_AUTO_EXPAND"
  ],
  "Tags": []
}) => Unable to fetch parameters [X] from parameter store for this account. (code=ValidationError)

UpdateParameters.template.json after toggling parameterName to Y (Conditions redacted):

{
 "Parameters": {
  "XParameter": {
   "Type": "AWS::SSM::Parameter::Value<String>",
   "Default": "Y"
  },
  "BootstrapVersion": {
   "Type": "AWS::SSM::Parameter::Value<String>",
   "Default": "/cdk-bootstrap/hnb659fds/version",
   "Description": "Version of the CDK Bootstrap resources in this environment, automatically retrieved from SSM Parameter Store. [cdk:skip]"
  }
 },
 "Resources": {
  "Z1BAC85DC": {
   "Type": "AWS::SSM::Parameter",
   "Properties": {
    "Type": "String",
    "Value": {
     "Ref": "XParameter"
    }
   },
   "Metadata": {
    "aws:cdk:path": "UpdateParameters/Z/Resource"
   }
  },
  "CDKMetadata": {
   "Type": "AWS::CDK::Metadata",
   "Properties": {
    "Analytics": "v2:deflate64:H4sIAAAAAAAA/03LUQ6CMBAE0LPw3y6WqBfgAgQPYGq7mgW6TXaLfhjuLoRE/ZqZ5E0DzRFc5V9qQxztRDd4X4oPo2nv3HnxCQuK6VHzLAHNCq+qaUNC/PiJf75s63tZe5s5UqHMi+EcEQatn+4M7gSHalAiKzMXSgj9nh/U0bnLlgAAAA=="
   },
   "Metadata": {
    "aws:cdk:path": "UpdateParameters/CDKMetadata/Default"
   },
   "Condition": "CDKMetadataAvailable"
  }
 },
 "Conditions": {},
 "Rules": {
  "CheckBootstrapVersion": {
   "Assertions": [
    {
     "Assert": {
      "Fn::Not": [
       {
        "Fn::Contains": [
         [
          "1",
          "2",
          "3",
          "4",
          "5"
         ],
         {
          "Ref": "BootstrapVersion"
         }
        ]
       }
      ]
     },
     "AssertDescription": "CDK bootstrap stack version 6 required. Please run 'cdk bootstrap' with a recent version of the CDK CLI."
    }
   ]
  }
 }
}

CDK CLI Version

2.24.1 (build 585f9ca)

Framework Version

No response

Node.js Version

v16.15.0

OS

macOS Monterey Version 12.4

Language

Typescript

Language Version

4.6.4

Other information

No response

nikovirtala commented 2 years ago

I guess this is more a @aws-cdk/core than @aws-cdk/aws-ssm issue because it is related to the CloudFormation ChangeSet creation.

tejasmr commented 2 years ago

const x = aws_ssm.StringParameter.fromStringParameterAttributes(this, 'X', {
  parameterName: parameterName,
}).stringValue;

Here the second parameter id is 'X', when you run aws ssm delete-parameter --name "X" you are deleting the parameter with id 'X', not parameter with name 'X'. Try changing parameterId to get your desired result:

const parameterName = 'Y';
const parameterId = 'Y';
const x = aws_ssm.StringParameter.fromStringParameterAttributes(this, parameterId, {
  parameterName: parameterName,
}).stringValue;

instead.

nikovirtala commented 2 years ago

@tezz-io I don't want to (in some cases even cannot) change the parameter construct Id, which causes a re-creation, I want to update the currently existing parameter value.

gshpychka commented 2 years ago

I want to update the currently existing parameter value.

You mean the imported parameter name, not the value, right?

nikovirtala commented 2 years ago

You mean the imported parameter name, not the value, right?

The Cloudformation parameter value (which is the SSM Parameter name).

peterwoodworth commented 2 years ago

We have the --no-previous-parameters option when deploying which will unblock here. Does this work for you?

gshpychka commented 2 years ago

@peterwoodworth why do you think this is not a bug?

peterwoodworth commented 2 years ago

I don't think anything is happening here that's unexpected - See CloudFormation docs on SSM Parameter types and CDK docs on parameters

CDK will use previous parameter values by default - and CloudFormation will use the previous SSM parameter key if no value is specified. Here, no parameter value is being supplied to the parameter that is only having its default value be changed, so it will remain the same between deployments.

We need to clarify this functionality in the SSM docs

gshpychka commented 2 years ago

CDK will use previous parameter values by default - and CloudFormation will use the previous SSM parameter key if no value is specified. Here, no parameter value is being supplied to the parameter that is only having its default value be changed, so it will remain the same between deployments.

But that's an implementation detail. When you consider the code in the issue:

    const x = aws_ssm.StringParameter.fromStringParameterAttributes(this, 'X', {
      parameterName: parameterName,
    }).stringValue;

The fact that the parameterName is implemented as a Parameter Type with a default value is an implementation detail. It makes sense that when a user changes the value of parameterName, they expect the construct to refer to a different parameter. The fact that it doesn't is a bug, in my opinion. The fact that it's the expected behavior due to the current implementation is just the cause of the bug, isn't it?

peterwoodworth commented 2 years ago

It makes sense that when a user changes the value of parameterName, they expect the construct to refer to a different parameter

I agree with you that this is a reasonable expectation to have, and ideally it should work this way. We want to abstract ourselves from CloudFormation right? Thanks for pushing back here

I've been struggling to think of how we could work around this due to being bound by CloudFormation, but I think there's a couple potential options that would need a bit more research, and I'm not completely sure if either option is viable:

All SSM parameter types no longer use the previous value by default with CDK (breaking change)
Somehow specify parameters created with this module to no longer use previous value by default (breaking change)

Since there's a pretty simple workaround here and the reason for this isn't obvious without some research, we should absolutely call this out in the docs anyway.

aws / aws-cdk