search

aws / aws-cdk

The AWS Cloud Development Kit is a framework for defining cloud infrastructure in code
https://aws.amazon.com/cdk
Apache License 2.0
11.7k stars 3.93k forks source link

cdk import picking false changes for NodejsFunction #31999

Open mrlikl opened 3 weeks ago

mrlikl commented 3 weeks ago

Describe the bug

When cdk import is comparing the current template with the synth template, it is picking false changes for NodejsFunction. For example, the below code 

const rdsSchemaInitializerFunction = new _lambda_node.NodejsFunction(
      this,
      'rds-schema-initializer',
      {
        bundling: {
          minify: true,
          sourceMap: false,
          sourcesContent: false,
          target: 'ES2020'
        },
        runtime: _lambda.Runtime.NODEJS_18_X,
        entry: path.join(__dirname, 'lambda/index.ts')
      }
    )

During deploy time will have the template: 

  rdsschemainitializerC99E4CA7:
    Type: AWS::Lambda::Function
    Properties:
      Code:
        S3Bucket:
          Fn::Sub: cdk-hnb659fds-assets-${AWS::AccountId}-${AWS::Region}
        S3Key: 336c1eb9382f4c0e4c84a0a0ef4cadd5d8df8a4921a98338abedd449e7cdaf12.zip
    Metadata:
      aws:cdk:path: CdktsStack/rds-schema-initializer/Resource
      aws:asset:path: asset.336c1eb9382f4c0e4c84a0a0ef4cadd5d8df8a4921a98338abedd449e7cdaf12
      aws:asset:is-bundled: true
      aws:asset:property: Code

and during synth of cdk import is having 

"rdsschemainitializerC99E4CA7": {
            "Type": "AWS::Lambda::Function",
            "Properties": {
                "Code": {
                    "S3Bucket": {
                        "Fn::Sub": "cdk-hnb659fds-assets-${AWS::AccountId}-${AWS::Region}"
                    },
                    "S3Key": "eae9438382815864ae642eab331faf132022d63ea8625cc80b327506fbe57216.zip"
                }, 
            "Metadata": {
                "aws:cdk:path": "CdktsStack/rds-schema-initializer/Resource",
                "aws:asset:path": "/Users/muralikl/Downloads/nov3/cdkts",
                "aws:asset:is-bundled": true,
                "aws:asset:property": "Code"
            }
        },

There is no change in the code, but can see that the s3 key is changing for the time being and asset path is also pointing to the file path.

Performing cdk import with --force will ignore this issue and will successfully import the resources.

Regression Issue

Last Known Working CDK Version

No response

Expected Behavior

cdk import to pick only the final template after synth and compare with the current deployed template

Current Behavior

cdk throwing a false positive change alongside import

Reproduction Steps

import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as _lambda_node from 'aws-cdk-lib/aws-lambda-nodejs';
import * as _lambda from 'aws-cdk-lib/aws-lambda';
import * as s3 from 'aws-cdk-lib/aws-s3';
import path = require('path/posix')

export class CdktsStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const rdsSchemaInitializerFunction = new _lambda_node.NodejsFunction(
      this,
      'rds-schema-initializer',
      {
        bundling: {
          minify: true,
          sourceMap: false,
          sourcesContent: false,
          target: 'ES2020'
        },
        runtime: _lambda.Runtime.NODEJS_18_X,
        entry: path.join(__dirname, 'lambda/index.ts')
      }
    )
   // deploy the above code first and then add the below code to run cdk import
    new s3.Bucket(this, 'test', {
      removalPolicy: cdk.RemovalPolicy.DESTROY
    })
  }
}

Possible Solution

No response

Additional Information/Context

No response

CDK CLI Version

2.165.0

Framework Version

No response

Node.js Version

20.17.0

OS

macos

Language

TypeScript

Language Version

No response

Other information

No response

khushail commented 2 weeks ago

Hi @mrlikl , thanks for reaching out.

The issue is not reproducible at my end. Sharing the steps and instructions which I did -

  1. Synthesized and deployed this code -
    const rdsSchemaInitializerFunction = new _lambda_node.NodejsFunction(
      this,
      'rds-schema-initializer',
      {
        bundling: {
          minify: true,
          sourceMap: false,
          sourcesContent: false,
          target: 'ES2020'
        },
        runtime: _lambda.Runtime.NODEJS_18_X,
        entry: path.join(__dirname, 'lambda/index.ts')
      }
    )

which produced this template -

"rdsschemainitializerC99E4CA7": {
      "Type": "AWS::Lambda::Function",
      "Properties": {
        "Code": {
          "S3Bucket": {
            "Fn::Sub": "cdk-hnb659fds-assets-${AWS::AccountId}-${AWS::Region}"
          },
          "S3Key": "e0690592dd9efac22e9fedd9ad94ef31d0e6b404bbf9142b33bba636f8bd9287.zip"
        },
        "Handler": "index.handler",
        "Role": {
          "Fn::GetAtt": [
            "rdsschemainitializerServiceRoleBA36691A",
            "Arn"
          ]
        },
        "Runtime": "nodejs18.x"
      },
      "DependsOn": [
        "rdsschemainitializerServiceRoleBA36691A"
      ],
      "Metadata": {
        "aws:cdk:path": "CdkImportIssueStack/rds-schema-initializer/Resource",
        "aws:asset:path": "asset.e0690592dd9efac22e9fedd9ad94ef31d0e6b404bbf9142b33bba636f8bd9287",
        "aws:asset:is-bundled": true,
        "aws:asset:property": "Code"
      }
    },

then I added this section of code -

    new s3.Bucket(this, 'test', {
      removalPolicy: cdk.RemovalPolicy.DESTROY
    })

which synthesized into this -

 "rdsschemainitializerC99E4CA7": {
   "Type": "AWS::Lambda::Function",
   "Properties": {
    "Code": {
     "S3Bucket": {
      "Fn::Sub": "cdk-hnb659fds-assets-${AWS::AccountId}-${AWS::Region}"
     },
     "S3Key": "e0690592dd9efac22e9fedd9ad94ef31d0e6b404bbf9142b33bba636f8bd9287.zip"
    },
    "Handler": "index.handler",
    "Role": {
     "Fn::GetAtt": [
      "rdsschemainitializerServiceRoleBA36691A",
      "Arn"
     ]
    },
    "Runtime": "nodejs18.x"
   },
   "DependsOn": [
    "rdsschemainitializerServiceRoleBA36691A"
   ],
   "Metadata": {
    "aws:cdk:path": "CdkImportIssueStack/rds-schema-initializer/Resource",
    "aws:asset:path": "asset.e0690592dd9efac22e9fedd9ad94ef31d0e6b404bbf9142b33bba636f8bd9287",
    "aws:asset:is-bundled": true,
    "aws:asset:property": "Code"
   }
  },
  "testAF53AC38": {
   "Type": "AWS::S3::Bucket",
   "UpdateReplacePolicy": "Delete",
   "DeletionPolicy": "Delete",
   "Metadata": {
    "aws:cdk:path": "CdkImportIssueStack/test/Resource"
   }
  },

Running cdk diff shows this result -

Screenshot 2024-11-04 at 3 21 11 PM

So in this case, no BucketKey or path is replaced or changed at all.

Could you please share more information on how to repro this scenario?

Thanks

mrlikl commented 2 weeks ago

Hey @khushail, thank you for taking a look. cdk diff will not have any changes. Do run a cdk import -v, you would get the warning / errors.

khushail commented 2 weeks ago

@mrlikl , thanks for sharing that information. I am able to repro the scenario and can see the asset:path being updated to local directory and s3 key also changing -

"rdsschemainitializerC99E4CA7": {
   "Type": "AWS::Lambda::Function",
   "Properties": {
    "Code": {
     "S3Bucket": {
      "Fn::Sub": "cdk-hnb659fds-assets-${AWS::AccountId}-${AWS::Region}"
     },
     "S3Key": "25c1c6edef08cbc19df7e2d16cc9e48e38ca6b33cd05c21ed00741c8754ecaad.zip"
    },
    "Handler": "index.handler",
    "Role": {
     "Fn::GetAtt": [
      "rdsschemainitializerServiceRoleBA36691A",
      "Arn"
     ]
    },
    "Runtime": "nodejs18.x"
   },
   "DependsOn": [
    "rdsschemainitializerServiceRoleBA36691A"
   ],
   "Metadata": {
    "aws:cdk:path": "CdkImportIssueStack/rds-schema-initializer/Resource",
    "aws:asset:path": "/Users/khushail/cdkImportIssue",
    "aws:asset:is-bundled": true,
    "aws:asset:property": "Code"
   }
  },

However I found a somewhat similar issue opened in the past -https://github.com/aws/aws-cdk/issues/31677. on diving deep into the code implementation. here is what I observed -

  1. Metadata support for aws:assets:path was introduced to support the SAM CLI I to find local assets used by resources in the template. here is an RFC which clearly mentions about adding these Metadata properties during bundling and using lambda function. https://github.com/aws/aws-cdk/blob/18c19fd49f2b83fee3d1cdb7de8b53ea310729b4/design/code-asset-metadata.md?plain=1#L25C1-L40C4
  2. This statement also mentions how assets path is used - https://github.com/aws/aws-cdk/blob/18c19fd49f2b83fee3d1cdb7de8b53ea310729b4/packages/aws-cdk-lib/aws-s3-assets/README.md?plain=1#L176

so when I am doing cdk import , I see these flag values added -

  'aws:cdk:enable-path-metadata': true,
  'aws:cdk:enable-asset-metadata': true,
  'aws:cdk:version-reporting': true,
  'aws:cdk:bundling-stacks': []

which I think are causing the asset path to change.

However I would reach out to team for getting their insights on this issue and share inputs here as well.

rix0rrr commented 1 week ago

Good debugging @khushail, it's definitely about the flags that are being used for synth. Apparently the flags are different between and import-synth and a deploy-synth.

That shouldn't be the case.

However, since this is an issue that affects not too many people and has a clear workaround (--force as mentioned in the report), I'm marking it as p2. Very happy to have this solved, but probably not something we'll prioritize any time soon.