aws / aws-cdk

The AWS Cloud Development Kit is a framework for defining cloud infrastructure in code
https://aws.amazon.com/cdk
Apache License 2.0
11.36k stars 3.77k forks source link

(cli) cdk import : resource-mapping options doesn't work #30318

Open sudhirkumar04 opened 1 month ago

sudhirkumar04 commented 1 month ago

Describe the bug

cdk import --resource-mapping=resource-mapping.json doesn't work. This returns

Unrecognized resource identifiers in mapping file: ImportedKMSKey1, ImportedKMSKey2
No resources selected for import.

ImportedKMSKey1, ImportedKMSKey2 are resource names

--resource-mapping option also return same result when it used after --record-resource-mapping

cdk import --record-resource-mapping=resource-mapping.json // this command prompted for keyId twice cdk import -resource-mapping=resource-mapping.json

This returns same as above

CoreAwsInfraStack
CoreAwsInfraStack/ImportedKMSKey1D83208A7/Resource: skipping
CoreAwsInfraStack/ImportedKMSKey2B3E62479/Resource: skipping
Unrecognized resource identifiers in mapping file: ImportedKMSKey1D83208A7, ImportedKMSKey2B3E62479
No resources selected for import.

Expected Behavior

cdk import --resource-mapping=resource-mapping.json should import the resources mentioned in the resource-mapping.json file

Current Behavior

cdk import --resource-mapping=resource-mapping.json doesn't work. This returns

CoreAwsInfraStack
CoreAwsInfraStack/ImportedKMSKey1/Resource: skipping
CoreAwsInfraStack/ImportedKMSKey2/Resource: skipping
Unrecognized resource identifiers in mapping file: ImportedKMSKey1, ImportedKMSKey2
No resources selected for import.

ImportedKMSKey1, ImportedKMSKey2 are resource names

Reproduction Steps

  1. Create a resource-mapping file, in this test, resource-mapping file is created inside lib/config/dev.json. E.g. below

    {
    "ImportedKMSKey1": {
      "KeyId": "xxxx-ad04-42ae-9b36-xxxxxxx"
    },
    "ImportedKMSKey2": {
      "KeyId": "xxxxx-dca0-4d07-9fe7-xxxxxx"
    }
    }
  2. Create file inside lib directory, this is for importing two kms keys.

    
    import * as cdk from 'aws-cdk-lib';
    import { Construct } from 'constructs';
    import { Key } from 'aws-cdk-lib/aws-kms'
    import {Tags} from 'aws-cdk-lib'
    import * as fs from 'fs';

export class CoreAwsInfraStack extends cdk.Stack { constructor(scope: Construct, id: string, props?: cdk.StackProps) { super(scope, id, props); const config = JSON.parse(fs.readFileSync(lib/config/dev.json, 'utf8')); for (const [key] of Object.entries(config)) { new Key(this, key, { }); } } }

3. Create a file inside bin directory

!/usr/bin/env node

import 'source-map-support/register'; import * as cdk from 'aws-cdk-lib'; import { CoreAwsInfraStack } from '../lib/core-aws-infra-stack';

const app = new cdk.App(); new CoreAwsInfraStack(app, 'CoreAwsInfraStack', { });


4. AWS_DEFAULT_PROFILE is set 
5. Run `cdk import --resource-mapping=lib/config/dev.json`

### Possible Solution

cdk import works but it prompts the kms key id which is a pain for rolling out for different accounts and environment. In order to avoid prompting keyId, resource-mapping should allow to map the resource. 

### Additional Information/Context

_No response_

### CDK CLI Version

2.122

### Framework Version

_No response_

### Node.js Version

21.7.1

### OS

mac

### Language

TypeScript

### Language Version

_No response_

### Other information

_No response_
khushail commented 1 month ago

@sudhirkumar04 , Thanks for reporting this.

I am able to repro the situation and resources are not getting imported however in my case, the error is a bit different which only says Skipping import. I found a somewhat similar past issue , there is a workaround suggested, not really sure if that works. You could check that out. Please feel free to share if that works!

I am currently investigating the root cause and looking into it. Will share my findings soon if there is an alternate on how this could be achieved.

sudhirkumar04 commented 1 month ago

thanks for looking into it @khushail Skipping import means its not importing the resource with the --resource-mapping options. The workaround you suggested has been checked before this issue raised. One of the solution suggested to use RemovalPolicy.DESTROY/RETAIN has also been checked. In the issue you suggested not using --resource-mapping to map the resource. The main purpose of using --resource-mapping option is to import resources using ci/cd where the resources is taken from a file rather entering details for each resource manually in the terminal

khushail commented 1 month ago

@sudhirkumar04 , I understood your point in providing the resources through the file and I tried to repro the same scenario and faced similar error -

Screenshot 2024-05-28 at 1 33 02 PM

As per my investigation, this code leads to reading the data from the file -

https://github.com/aws/aws-cdk/blob/ba8edb3e16e3b7c72bb2a38bb3318969e0e9f054/packages/aws-cdk/lib/cdk-toolkit.ts#L551

which is invoking this module -

https://github.com/aws/aws-cdk/blob/ba8edb3e16e3b7c72bb2a38bb3318969e0e9f054/packages/aws-cdk/lib/import.ts#L92

Looks like, the file is not being read properly and error is produced by contents being read as Unknown - https://github.com/aws/aws-cdk/blob/ba8edb3e16e3b7c72bb2a38bb3318969e0e9f054/packages/aws-cdk/lib/import.ts#L111

Since the direct option of passing the logical id exists through cdk import, marking this as P2.