aws / aws-cdk

The AWS Cloud Development Kit is a framework for defining cloud infrastructure in code
https://aws.amazon.com/cdk
Apache License 2.0
11.63k stars 3.91k forks source link

(custom-resources): Package @aws-sdk/client-memory-db does not exist #31755

Open irwinp-p opened 1 week ago

irwinp-p commented 1 week ago

Describe the bug

I'm using AwsCustomResource from typescipt to perform an AwsSdkCall with the memorydb service. I set the AwsSdkCall.service to be '@aws-sdk/client-memorydb' when creating the custom resource. However this server gets incorrectly mapped to '@aws-sdk/client-memory-db' for which no package exists.

This causes the deployment of the custom resource to fail.

Regression Issue

Last Known Working CDK Version

No response

Expected Behavior

The custom resource should deploy successfully

Current Behavior

The deployment fails with error message from the lambda Error: Package @aws-sdk/client-memory-db does not exist. at qe (/var/task/index.js:1:125653) at Object.De (/var/task/index.js:1:126372) at le (/var/task/index.js:1:127784) at Runtime.He [as handler] (/var/task/index.js:1:127887) at Runtime.handleOnceNonStreaming (file:///var/runtime/index.mjs:1173:29)

Reproduction Steps

Use this custom resource construct to reproduce

new AwsCustomResource(this, 'UpdateCluster', {
      timeout: cdk.Duration.minutes(1),
      installLatestAwsSdk: false,
      onUpdate: {
        service: '@aws-sdk/client-memorydb',
        action: 'UpdateCluster',
        parameters: {
          ClusterName: <REPLACE_WITH_MEMORYDB_CLUSTER_NAME>,
          Engine: 'valkey',
        },
        physicalResourceId: PhysicalResourceId.of(`${id}-${this.node.addr.slice(0, 16)}`),
      },
      policy: AwsCustomResourcePolicy.fromStatements([
        new cdk.aws_iam.PolicyStatement({
          effect: cdk.aws_iam.Effect.ALLOW,
          actions: ['memorydb:UpdateCluster'],
          resources: ['*'],
        }),
      ]),
    });
  }

Possible Solution

The entry here https://github.com/aws/aws-cdk/blob/78a7182c2a262fe4f635deb103be8757e5a20efc/packages/%40aws-cdk/aws-custom-resource-sdk-adapter/lib/sdk-v2-to-v3.json#L94 appears to be incorrect. The package name in AWS Javascript SDK v3 for MemoryDB is " @aws-sdk/client-memorydb" and not "@aws-sdk/client-memory-db" https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/client/memorydb/

Additional Information/Context

No response

CDK CLI Version

2.155.0

Framework Version

No response

Node.js Version

v20

OS

MacOSX

Language

TypeScript

Language Version

No response

Other information

No response

ashishdhingra commented 1 week ago

Reproducible using code below:

import * as cdk from 'aws-cdk-lib';
import { AwsCustomResource, AwsCustomResourcePolicy, PhysicalResourceId } from 'aws-cdk-lib/custom-resources';

export class CdktestStack extends cdk.Stack {
  constructor(scope: cdk.App, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    new AwsCustomResource(this, 'UpdateCluster', {
      timeout: cdk.Duration.minutes(1),
      installLatestAwsSdk: false,
      onUpdate: {
        service: '@aws-sdk/client-memorydb',
        action: 'UpdateCluster',
        parameters: {
          ClusterName: 'test-memorydb-cluster',
          Engine: 'Valkey',
        },
        physicalResourceId: PhysicalResourceId.of(`${id}-${this.node.addr.slice(0, 16)}`),
      },
      policy: AwsCustomResourcePolicy.fromStatements([
        new cdk.aws_iam.PolicyStatement({
          effect: cdk.aws_iam.Effect.ALLOW,
          actions: ['memorydb:UpdateCluster'],
          resources: ['*'],
        }),
      ]),
    });
  }
}

Deploying it using cdk deploy gives the below error in CloudFormation:

✨  Synthesis time: 4.36s

CdktestStack: start: Building 4ac73a2ad39760cf942fa8196c4d1807ecb6062c0baf25176c20e0ac5dd87121:<<ACCOUNT-ID>>-us-east-2
CdktestStack: success: Built 4ac73a2ad39760cf942fa8196c4d1807ecb6062c0baf25176c20e0ac5dd87121:<<ACCOUNT-ID>>-us-east-2
CdktestStack: start: Publishing 4ac73a2ad39760cf942fa8196c4d1807ecb6062c0baf25176c20e0ac5dd87121:<<ACCOUNT-ID>>-us-east-2
CdktestStack: success: Published 4ac73a2ad39760cf942fa8196c4d1807ecb6062c0baf25176c20e0ac5dd87121:<<ACCOUNT-ID>>-us-east-2
Stack undefined
This deployment will make potentially sensitive changes according to your current security approval level (--require-approval broadening).
Please confirm you intend to make the following modifications:

IAM Statement Changes
┌───┬────────────────────────────────────────────────────────┬────────┬────────────────────────┬────────────────────────────────────────────────────────┬───────────┐
│   │ Resource                                               │ Effect │ Action                 │ Principal                                              │ Condition │
├───┼────────────────────────────────────────────────────────┼────────┼────────────────────────┼────────────────────────────────────────────────────────┼───────────┤
│ + │ ${AWS679f53fac002430cb0da5b7982bd2287/ServiceRole.Arn} │ Allow  │ sts:AssumeRole         │ Service:lambda.amazonaws.com                           │           │
├───┼────────────────────────────────────────────────────────┼────────┼────────────────────────┼────────────────────────────────────────────────────────┼───────────┤
│ + │ *                                                      │ Allow  │ memorydb:UpdateCluster │ AWS:${AWS679f53fac002430cb0da5b7982bd2287/ServiceRole} │           │
└───┴────────────────────────────────────────────────────────┴────────┴────────────────────────┴────────────────────────────────────────────────────────┴───────────┘
IAM Policy Changes
┌───┬────────────────────────────────────────────────────┬────────────────────────────────────────────────────────────────────────────────┐
│   │ Resource                                           │ Managed Policy ARN                                                             │
├───┼────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────────────┤
│ + │ ${AWS679f53fac002430cb0da5b7982bd2287/ServiceRole} │ arn:${AWS::Partition}:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole │
└───┴────────────────────────────────────────────────────┴────────────────────────────────────────────────────────────────────────────────┘
(NOTE: There may be security-related changes not in this list. See https://github.com/aws/aws-cdk/issues/1299)

Do you wish to deploy these changes (y/n)? y
CdktestStack: deploying... [1/1]
CdktestStack: creating CloudFormation changeset...
1:18:15 PM | CREATE_FAILED        | Custom::AWS           | UpdateCluster5DD573A1
Received response status [FAILED] from custom resource. Message returned: Package @aws-sdk/client-memory-db does not exist. (RequestId: 50b82c83-4dfe-4d87-956c-b07bbb28625e)

❌  CdktestStack failed: The stack named CdktestStack failed creation, it may need to be manually deleted from the AWS console: ROLLBACK_COMPLETE: Received response status [FAILED] from custom resource. Message returned: Package @aws-sdk/client-memory-db does not exist. (RequestId: 50b82c83-4dfe-4d87-956c-b07bbb28625e)

Findings:

ashishdhingra commented 2 days ago

Looks like the below files are auto-generated from scripts/update-sdkv3-parameters-model.ts:

Per script update-sdkv3-parameters-model.sh, it appears to download https://github.com/aws/aws-sdk-js-v3/archive/refs/heads/main.zip. This file appears to be archived (even though it's downloadable). The v3Name per code here should have been memorydb per latest downloaded file.

EDIT: Looks like there is more to it. writeV2ToV3Mapping() reads aws-sdk-js-codemod/dist/transforms/v2-to-v3/config/CLIENT_PACKAGE_NAMES_MAP, which is replacing MemoryDB from CLIENT_NAMES.ts to memory-db here. Hence, when the script update-sdkv3-parameters-model.sh is run manually, it doesn't change the mapping. We would first need to reach out to JS SDK team to add replace() override for memory-db similar to what is being done for some other services here.

Bug https://github.com/aws/aws-sdk-js-codemod/issues/949 opened. It's fixed via PR https://github.com/aws/aws-sdk-js-codemod/pull/951 and released in aws-sdk-js-codemod 2.3.3.

Running ./scripts/update-sdkv3-parameters-model.sh removes memorydb per logic mapping since key-value are same. This is fine since normalizeServiceName() extracts the service name and calls v2ToV3Mapping() here, it would use original service name and would work fine (per findings in https://github.com/aws/aws-cdk/issues/31755#issuecomment-2414930902).

ashishdhingra commented 1 day ago

Possible PR contribution steps: