aws / aws-cdk

The AWS Cloud Development Kit is a framework for defining cloud infrastructure in code
https://aws.amazon.com/cdk
Apache License 2.0
11.52k stars 3.86k forks source link

(@aws-cdk/cloudformation-include): including a valid template fails #16889

Closed mcuervoe closed 2 years ago

mcuervoe commented 2 years ago

What is the problem?

Including a valid cloudformation template provided by cognitect/datomic using @aws-cdk/cloudformation-include fails

Reproduction Steps

The following code fails

import * as cdk from '@aws-cdk/core';
import * as cfninc from '@aws-cdk/cloudformation-include';

export class CdkTest2Stack extends cdk.Stack {
  constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    // The code that defines your stack goes here
    const template = new cfninc.CfnInclude(this, 'Template', {
      templateFile: 'datomic-compute-936-9118.json',
    });
  }
}

The error that I get is

$ cdk ls
Invalid tag input expected {key, value, propagateAtLaunch} have "<unresolved-token>"
....

The template can be found here https://s3.amazonaws.com/datomic-cloud-1/cft/936-9118/datomic-compute-936-9118.json or in the document attached

datomic-compute-936-9118.zip

What did you expect to happen?

The template should be included without errors

What actually happened?

The complete error message trace after executing cdk ls

Invalid tag input expected {key, value, propagateAtLaunch} have "<unresolved-token>"

/home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/aws-autoscaling/lib/autoscaling.generated.ts:381
            tags: this.tags.renderTags(),
                            ^
While synthesizing CdkTest2Stack/Template/TxAutoScalingGroup: Cannot read property 'renderTags' of undefined
  --- resource created at ---
  at Function._fromCloudFormation (/home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/aws-autoscaling/lib/autoscaling.generated.ts:231:21)
  at CfnInclude.getOrCreateResource (/home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/cloudformation-include/lib/cfn-include.ts:519:40)
  at Object.findResource (/home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/cloudformation-include/lib/cfn-include.ts:487:21)
  at Object.findRefTarget (/home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/cloudformation-include/lib/cfn-include.ts:495:21)
  at CfnParser.parseIfCfnIntrinsic (/home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/cloudformation-include/node_modules/@aws-cdk/core/lib/cfn-parse.ts:523:42)
  at CfnParser.parseValue (/home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/cloudformation-include/node_modules/@aws-cdk/core/lib/cfn-parse.ts:494:33)
  at /home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/cloudformation-include/node_modules/@aws-cdk/core/lib/cfn-parse.ts:490:38
  at Array.map (<anonymous>)
  at CfnParser.parseValue (/home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/cloudformation-include/node_modules/@aws-cdk/core/lib/cfn-parse.ts:490:23)
  at CfnParser.parseValue (/home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/cloudformation-include/node_modules/@aws-cdk/core/lib/cfn-parse.ts:500:25)
  at Function._fromCloudFormation (/home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/aws-codedeploy/lib/codedeploy.generated.ts:692:51)
  at CfnInclude.getOrCreateResource (/home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/cloudformation-include/lib/cfn-include.ts:519:40)
  at Object.findResource (/home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/cloudformation-include/lib/cfn-include.ts:487:21)
  at Object.findRefTarget (/home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/cloudformation-include/lib/cfn-include.ts:495:21)
  at go (/home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/cloudformation-include/node_modules/@aws-cdk/core/lib/cfn-parse.ts:741:40)
  at go (/home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/cloudformation-include/node_modules/@aws-cdk/core/lib/cfn-parse.ts:711:52)
  at go (/home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/cloudformation-include/node_modules/@aws-cdk/core/lib/cfn-parse.ts:734:42)
  at go (/home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/cloudformation-include/node_modules/@aws-cdk/core/lib/cfn-parse.ts:734:42)
  at CfnParser.parseFnSubString (/home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/cloudformation-include/node_modules/@aws-cdk/core/lib/cfn-parse.ts:686:19)
  at CfnParser.parseIfCfnIntrinsic (/home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/cloudformation-include/node_modules/@aws-cdk/core/lib/cfn-parse.ts:649:21)
  at CfnParser.parseValue (/home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/cloudformation-include/node_modules/@aws-cdk/core/lib/cfn-parse.ts:494:33)
  at /home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/cloudformation-include/node_modules/@aws-cdk/core/lib/cfn-parse.ts:490:38
  at Array.map (<anonymous>)
  at CfnParser.parseValue (/home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/cloudformation-include/node_modules/@aws-cdk/core/lib/cfn-parse.ts:490:23)
  at CfnParser.parseValue (/home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/cloudformation-include/node_modules/@aws-cdk/core/lib/cfn-parse.ts:500:25)
  at /home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/cloudformation-include/node_modules/@aws-cdk/core/lib/cfn-parse.ts:490:38
  at Array.map (<anonymous>)
  at CfnParser.parseValue (/home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/cloudformation-include/node_modules/@aws-cdk/core/lib/cfn-parse.ts:490:23)
  at CfnParser.parseValue (/home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/cloudformation-include/node_modules/@aws-cdk/core/lib/cfn-parse.ts:500:25)
  at CfnParser.parseValue (/home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/cloudformation-include/node_modules/@aws-cdk/core/lib/cfn-parse.ts:500:25)
  at Function._fromCloudFormation (/home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/aws-iam/lib/iam.generated.ts:881:51)
  at CfnInclude.getOrCreateResource (/home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/cloudformation-include/lib/cfn-include.ts:519:40)
  at Object.findResource (/home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/cloudformation-include/lib/cfn-include.ts:487:21)
  at CfnParser.handleAttributes (/home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/cloudformation-include/node_modules/@aws-cdk/core/lib/cfn-parse.ts:375:39)
  at Function._fromCloudFormation (/home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/aws-lambda/lib/lambda.generated.ts:1702:24)
  at CfnInclude.getOrCreateResource (/home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/cloudformation-include/lib/cfn-include.ts:519:40)
  at Object.findResource (/home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/cloudformation-include/lib/cfn-include.ts:487:21)
  at CfnParser.parseIfCfnIntrinsic (/home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/cloudformation-include/node_modules/@aws-cdk/core/lib/cfn-parse.ts:549:36)
  at CfnParser.parseValue (/home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/cloudformation-include/node_modules/@aws-cdk/core/lib/cfn-parse.ts:494:33)
  at CfnParser.parseValue (/home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/cloudformation-include/node_modules/@aws-cdk/core/lib/cfn-parse.ts:500:25)
  at /home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/cloudformation-include/node_modules/@aws-cdk/core/lib/cfn-parse.ts:490:38
  at Array.map (<anonymous>)
  at CfnParser.parseValue (/home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/cloudformation-include/node_modules/@aws-cdk/core/lib/cfn-parse.ts:490:23)
  at CfnParser.parseIfCfnIntrinsic (/home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/cloudformation-include/node_modules/@aws-cdk/core/lib/cfn-parse.ts:638:28)
  at CfnParser.parseValue (/home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/cloudformation-include/node_modules/@aws-cdk/core/lib/cfn-parse.ts:494:33)
  at CfnParser.parseValue (/home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/cloudformation-include/node_modules/@aws-cdk/core/lib/cfn-parse.ts:500:25)
  at Function._fromCloudFormation (/home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/aws-stepfunctions/lib/stepfunctions.generated.ts:315:51)
  at CfnInclude.getOrCreateResource (/home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/cloudformation-include/lib/cfn-include.ts:519:40)
  at new CfnInclude (/home/miguel/git/compnsate/cdk-test2/node_modules/@aws-cdk/cloudformation-include/lib/cfn-include.ts:97:12)
  at new CdkTest2Stack (/home/miguel/git/compnsate/cdk-test2/lib/cdk-test2-stack.ts:9:22)
  at Object.<anonymous> (/home/miguel/git/compnsate/cdk-test2/bin/cdk-test2.ts:7:1)
  at Module._compile (internal/modules/cjs/loader.js:999:30)
  at Module.m._compile (/home/miguel/git/compnsate/cdk-test2/node_modules/ts-node/src/index.ts:1056:23)
  at Module._extensions..js (internal/modules/cjs/loader.js:1027:10)
  at Object.require.extensions.<computed> [as .ts] (/home/miguel/git/compnsate/cdk-test2/node_modules/ts-node/src/index.ts:1059:12)

CDK CLI Version

1.126.0 (build f004e1a)

Framework Version

1,126.0

Node.js Version

v12.22.6

OS

Linux

Language

Typescript, Python, Java

Language Version

Java (11), python (3.8.6), Typescript (3.9.7)

Other information

Although I have not provided the code, the same error message is generated in java and python equivalent versions

peterwoodworth commented 2 years ago

Here's where that error is being thrown in the code: https://github.com/aws/aws-cdk/blob/7966f8d48c4bff26beb22856d289f9d0c7e7081d/packages/%40aws-cdk/core/lib/tag-manager.ts#L54

I suspect that the CfnInclude class is getting confused by ASG having a tags property.

skinny85 commented 2 years ago

Thanks for opening the issue @mcuervoe. Confirming I was able to reproduce it. I'm investigating what is happening.

Interestingly, a minimum template like this:

{
    "Resources": {
        "TxAutoScalingGroup": {
            "Type": "AWS::AutoScaling::AutoScalingGroup",
            "Properties": {
                "Tags": [
                    {
                        "Key": "datomic:subscriber",
                        "Value": "SystemName",
                        "PropagateAtLaunch": true
                    }
                ],
                "MinSize": "1",
                "MaxSize": "3"
            }
        }
    }
}

does not reproduce the problem, so the issue must be more subtle.

skinny85 commented 2 years ago

Ah. This does though:

{
    "Conditions": {
        "ValcacheServerEnabled": true
    },
    "Resources": {
        "TxAutoScalingGroup": {
            "Type": "AWS::AutoScaling::AutoScalingGroup",
            "Properties": {
                "Tags": [
                    {
                        "Fn::If": [
                            "ValcacheServerEnabled",
                            {
                                "Key": "datomic:cache-group",
                                "Value": "SystemName",
                                "PropagateAtLaunch": true
                            },
                            {
                                "Ref": "AWS::NoValue"
                            }
                        ]
                    }
                ],
                "MinSize": "1",
                "MaxSize": "3"
            }
        }
    }
}

Let me see what the problem is.

skinny85 commented 2 years ago

Interesting... you actually cannot represent this template in the CDK L1!

This doesn't compile:

    const tagIfTrue: autoscaling.CfnAutoScalingGroup.TagPropertyProperty = {
      key: 'datomic:cache-group',
      value: 'SystemName',
      propagateAtLaunch: true,
    };
    new autoscaling.CfnAutoScalingGroup(this, 'CfnAutoScalingGroup', {
      minSize: '1',
      maxSize: '3',
      tags: [
        cdk.Fn.conditionIf('ValcacheServerEnabled', tagIfTrue, cdk.Aws.NO_VALUE),
      ],
    });

This is probably the reason why it fails including.

@mcuervoe can you edit the template before including it to remove the Fn::If there? Either remove it, if ValcacheServerEnabled is false, or leave just { "Key": "datomic:cache-group", "Value": { "Ref": "SystemName" }, "PropagateAtLaunch": true } if ValcacheServerEnabled is true? That should unblock you.

Looks like this might be a more fundamental problem with the CDK L1 layer than just a simple change in the CfnInclude class, unfortunately, so it might take us longer to fix.

github-actions[bot] commented 2 years ago

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.