aws / aws-cdk

The AWS Cloud Development Kit is a framework for defining cloud infrastructure in code
https://aws.amazon.com/cdk
Apache License 2.0
11.68k stars 3.93k forks source link

(cloudformation-include): CfnInclude misses converting reference when not preserving logical IDs #15705

Closed flemjame-at-amazon closed 3 years ago

flemjame-at-amazon commented 3 years ago

I thought it would be a neat idea to use AWS Distributed Load Testing through CDK, so I downloaded the template (link) and used CfnInclude like so.

export interface DistributedLoadTestingProps {
  adminUsername: string,
  adminEmail: string
}

export class DistributedLoadTesting extends Construct {

  constructor(scope: Construct, id: string, props: DistributedLoadTestingProps) {
    super(scope, id);
    const template = new CfnInclude(this, "DLTTemplate", {
      templateFile: 'distributed-load-testing-on-aws.template',
      preserveLogicalIds: false,
      parameters: {
        AdminEmail: props.adminEmail,
        AdminName: props.adminUsername,
        DockerHubSecret: ''
      }
    });
}

However, when I tried to deploy the stack, I got the following error:

Template format error: Unresolved resource dependencies [ScenariosBucket, Api] in the Resources block of the template

I've attached the original template file (the one I am CfnInclude-ing) and the synthesized template (after being CfnInclude-d) to this issue.

distributed-load-testing-on-aws.template.txt ## This template is vended by AWS and works really well synthesized-template-after-cfninclude.template.txt ## This template is the above template, but CfnInclude-d, and fails when trying to create a stack.

In the original template there are two resources, called ScenariosBucket and Api. I don't preserve logical IDs, so I expect them all to change, and sure enough they did. They changed to:

Before:

## Storage
  ScenariosBucket:
    DeletionPolicy: Retain
    UpdateReplacePolicy: Retain
    Type: AWS::S3::Bucket
    Properties:
      LoggingConfiguration:
        DestinationBucketName: !Ref LogsBucket
        LogFilePrefix: scenarios-bucket-access/
      BucketEncryption:
        ServerSideEncryptionConfiguration:
        - ServerSideEncryptionByDefault:
            SSEAlgorithm: aws:kms
## Keeps going but you get the idea

After

"DLTDLTTemplateScenariosBucket564214B9": {
      "Type": "AWS::S3::Bucket",
      "Properties": {
        "BucketEncryption": {
          "ServerSideEncryptionConfiguration": [
            {
## Again keeps going

Before:

  Api:
    Type: AWS::ApiGateway::RestApi
    Properties:
      Body:
        swagger: "2.0"
        info:

After:

    "DLTDLTTemplateApi1B17D6FA": {
      "Type": "AWS::ApiGateway::RestApi",
      "Properties": {
        "Body": {
          "swagger": "2.0",
          "info": {

However, there was a missed reference for each of these, which wasn't converted to the new logical ID, and which is causing the stack to fail to create. It is a custom resource, but I am not sure why these failed to convert -- the custom resource has other references that did successfully convert to new logical ids.

Original:

  ConsoleConfig:
    Type: Custom::CopyConsoleFiles
    Properties:
      ServiceToken: !GetAtt CustomResource.Arn
      Resource: ConfigFile
      DestBucket: !Ref ConsoleBucket
      AwsExports:
        !Sub |
          const awsConfig = {
            cw_dashboard: 'https://console.aws.amazon.com/cloudwatch/home?region=${AWS::Region}#dashboards:name=',
            ecs_dashboard: 'https://${AWS::Region}.console.aws.amazon.com/ecs/home?region=${AWS::Region}#/clusters/${AWS::StackName}/tasks',
            aws_project_region: '${AWS::Region}',
            aws_cognito_region: '${AWS::Region}',
            aws_cognito_identity_pool_id: '${CognitoIdentityPool}', ## This one succeeds
            aws_user_pools_id: '${CognitoUserPool}', ## This one succeeds
            aws_user_pools_web_client_id: '${CognitoUserPoolClient}', ## This one succeeds
            oauth: {},
            aws_cloud_logic_custom: [
              {
                name: 'dlts',
                endpoint: 'https://${Api}.execute-api.${AWS::Region}.amazonaws.com/prod', ## This one fails
                region: '${AWS::Region}'
              }
            ],
            aws_user_files_s3_bucket: '${ScenariosBucket}', ## This one fails
            aws_user_files_s3_bucket_region: '${AWS::Region}'
          }

After CfnInclude:

    "DLTDLTTemplateConsoleConfigF678F1BD": {
      "Type": "Custom::CopyConsoleFiles",
      "Properties": {
        "ServiceToken": {
          "Fn::GetAtt": "DLTDLTTemplateCustomResource47815FAC.Arn"
        },
        "Resource": "ConfigFile",
        "DestBucket": {
          "Ref": "DLTDLTTemplateConsoleBucket39E49E4A"
        },
        "AwsExports": {
          "Fn::Sub": "const awsConfig = { ## I formatted this part so it was more obvious
              cw_dashboard: 'https://console.aws.amazon.com/cloudwatch/home?region=${AWS::Region}#dashboards:name=',
              ecs_dashboard: 'https://${AWS::Region}.console.aws.amazon.com/ecs/home?region=${AWS::Region}#/clusters/${AWS::StackName}/tasks',
              aws_project_region: '${AWS::Region}',
              aws_cognito_region: '${AWS::Region}',
              aws_cognito_identity_pool_id: '${DLTDLTTemplateCognitoIdentityPool8FDF431D}', ## This converted correctly
              aws_user_pools_id: '${DLTDLTTemplateCognitoUserPoolE00202A7}', ## This converted correctly
              aws_user_pools_web_client_id: '${DLTDLTTemplateCognitoUserPoolClientF5DC291C}', ## This converted correctly
              oauth: {},
              aws_cloud_logic_custom: [
                {
                  name: 'dlts',
                  endpoint: 'https://${Api}.execute-api.${AWS::Region}.amazonaws.com/prod', ## This didn't convert correctly
                  region: '${AWS::Region}'
                }
              ],
              aws_user_files_s3_bucket: '${ScenariosBucket}', ## This didn't convert correctly
              aws_user_files_s3_bucket_region: '${AWS::Region}'
            }"
        }
      }
    },

Reproduction Steps

What did you expect to happen?

I should be able to create a stack using my template that has the CfnInclude in it.

What actually happened?

Got a CloudFormation error upon trying to create the stack:

Template format error: Unresolved resource dependencies [ScenariosBucket, Api] in the Resources block of the template

Environment

Other


This is :bug: Bug Report

flemjame-at-amazon commented 3 years ago

Deeper digging uncovered more instances of substitutions not happening:

"DLTDLTTemplateCognitoUserPoolE00202A7": {
      "Type": "AWS::Cognito::UserPool",
      "Properties": {
        "AdminCreateUserConfig": {
          "AllowAdminCreateUserOnly": true,
          "InviteMessageTemplate": {
            "EmailMessage": {
              "Fn::Sub": "<p>You are invited to join the Distribution Load Testing Solution.</p>\n<p>Username: <strong>{username}</strong></p>\n<p>Password: <strong>{####}</strong></p>\n<p>Console: <strong>https://${ConsoleCloudFront.DomainName}/<
/strong></p>\n" ## Should be ${DLTDLTTemplateConsoleCloudFrontCEAF1B6F} instead
            },
"DLTDLTTemplateTaskRunnerStepFunctionsAC1DF482": {
      "Type": "AWS::StepFunctions::StateMachine",
      "Properties": {
        "RoleArn": {
          "Fn::GetAtt": "DLTDLTTemplateTaskRunnerStepFunctionsRole97DD7EF6.Arn"
        },
        "DefinitionString": {
          "Fn::Sub": "{
  "Comment": "Distributed Load Testing on AWS Task Runner",
  "StartAt": "Check ECR is ready",
  "States": {
    "Check ECR is ready": {
      "Type": "Task",
      "Resource": "${DLTDLTTemplateEcrChecker53762B06.Arn}",
      "InputPath": "$",
      "OutputPath": "$",
      "Next": "ECR is ready?"
    },
    "ECR is ready?": {
      "Type": "Choice",
      "Choices": [
        {
          "Variable": "$.ecrReady",
          "BooleanEquals": true,
          "Next": "Check running tests"
        }
      ],
      "Default": "Wait 1 minute - ECR ready"
    },
    "Wait 1 minute - ECR ready": {
      "Comment": "Wait 1 minute to check ECR readiness",
      "Type": "Wait",
      "Seconds": 60,
      "Next": "Check ECR is ready"
    },
    "Check running tests": {
      "Type": "Task",
      "Resource": "${TaskStatusChecker.Arn}", ## Should be DLTDLTTemplateTaskStatusChecker327377ED.Arn
      "InputPath": "$",
      "OutputPath": "$",
      "Next": "No running test?"
    },
    "No running test?": {
      "Type": "Choice",
      "Choices": [
        {
          "Variable": "$.isRunning",
          "BooleanEquals": false,
          "Next": "Run workers"
        }
      ],
      "Default": "Test is still running"
    },
    "Test is still running": {
      "Type": "Fail",
      "Error": "TestAlreadyRunning",
      "Cause": "The same test is already running."
    },
    "Run workers": {
      "Type": "Task",
      "Resource": "${TaskRunner.Arn}", ## Should be DLTDLTTemplateTaskRunner8A4D7569.Arn
      "InputPath": "$",
      "OutputPath": "$",
      "Next": "All workers launched?"
    },
    "All workers launched?": {
      "Type": "Choice",
      "Choices": [
        {
          "Variable": "$.isRunning",
          "BooleanEquals": false,
          "Next": "Cancel Test"
        },
        {
          "Variable": "$.taskRunner.runTaskCount",
          "NumericEquals": 1,
          "Next": "Wait 1 minute - worker status"
        },
        {
          "Variable": "$.taskRunner.runTaskCount",
          "NumericEquals": 0,
          "Next": "Wait 1 minute - task status"
        }
      ],
      "Default": "Run workers"
    },
    "Cancel Test": {
      "Type": "Task",
      "Resource": "${TaskCanceler.Arn}", ## Should be DLTDLTTemplateTaskCanceler93036AAF.Arn
      "InputPath": "$",
      "ResultPath": null,
      "OutputPath": "$",
      "Next": "Parse result"
    },
    "Wait 1 minute - worker status": {
      "Comment": "Wait 1 minute to check task status again",
      "Type": "Wait",
      "Seconds": 60,
      "Next": "Check worker status"
    },
    "Check worker status": {
      "Type": "Task",
      "Resource": "${TaskStatusChecker.Arn}", ## Should be DLTDLTTemplateTaskStatusChecker327377ED.Arn
      "InputPath": "$",
      "OutputPath": "$",
      "Next": "All workers running?"
    },
    "All workers running?": {
      "Type": "Choice",
      "Choices": [
        {
          "Variable": "$.numTasksRunning",
          "NumericEqualsPath": "$.scenario.taskCount",
          "Next": "Run leader task"
        },
        {
          "Variable": "$.isRunning",
          "BooleanEquals": false,
          "Next": "Parse result"
        }
      ],
      "Default": "Wait 1 minute - worker status"
    },
    "Run leader task": {
      "Type": "Task",
      "Resource": "${TaskRunner.Arn}", ## Should be DLTDLTTemplateTaskRunner8A4D7569.Arn
      "InputPath": "$",
      "OutputPath": "$",
      "Next": "Wait 1 minute - task status"
    },
    "Wait 1 minute - task status": {
      "Comment": "Wait 1 minute to check task status again",
      "Type": "Wait",
      "Seconds": 60,
      "Next": "Check task status"
    },
    "Check task status": {
      "Type": "Task",
      "Resource": "${TaskStatusChecker.Arn}", ## Should be DLTDLTTemplateTaskStatusChecker327377ED.Arn
      "InputPath": "$",
      "OutputPath": "$",
      "Next": "All tasks done?"
    },
    "All tasks done?": {
      "Type": "Choice",
      "Choices": [
        {
          "Variable": "$.isRunning",
          "BooleanEquals": false,
          "Next": "Parse result"
        }
      ],
      "Default": "Wait 1 minute - task status"
    },
    "Parse result": {
      "Type": "Task",
      "Resource": "${ResultsParser.Arn}", ## Should be DLTDLTTemplateResultsParserA3D2FE0A
      "Next": "Done"
    },
    "Done": {
      "Type": "Pass",
      "End": true
    }
  }
}
"
        },

At a glance, a commonality appears to be the use of Sub!

skinny85 commented 3 years ago

Hi @flemjame-at-amazon ,

thanks for opening the issue, but both me and @comcalvi tried reproducing this locally, and it worked for both of us.

Can you create a GitHub repository with a sample CDK application that reproduces this problem, so that we can clone it, and see what happens?

Thanks, Adam

skinny85 commented 3 years ago

Also, I hope you made a typo, and you're not actually using CDK in version 1.12.0 😉.

flemjame-at-amazon commented 3 years ago

Also, I hope you made a typo, and you're not actually using CDK in version 1.12.0 😉.

Nope, it's actually 1.112.0, my mistake!

skinny85 commented 3 years ago

Turns out, the customer was using version 1.96.0 of the CDK, not 1.112.0. I confirm upgrading to 1.115.0 fixes the bug (this is most likely the same bug as was fixed in https://github.com/aws/aws-cdk/pull/14512, first released in CDK version 1.103.0).

Resolving, please comment if upgrading to 1.103.0 does not fix the problem.

github-actions[bot] commented 3 years ago

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.

flemjame-at-amazon commented 3 years ago

I can confirm this was an issue on my end -- my package-lock.json was on 1.95.1.