aws / aws-cdk

The AWS Cloud Development Kit is a framework for defining cloud infrastructure in code
https://aws.amazon.com/cdk
Apache License 2.0
11.35k stars 3.76k forks source link

rds: Upgrade(2.142.0 -> 2.143.0) added an explicit dependency between some of our DB constructs. #30383

Open mamta925 opened 1 month ago

mamta925 commented 1 month ago

Describe the bug

In CDK upgrade Package Type Update Change
aws-cdk (source) devDependencies minor 2.142.0 -> 2.143.0
aws-cdk-lib (source) dependencies minor 2.142.0 -> 2.143.0

This upgrade added an explicit dependency between some of our DB constructs.

"DbDbClusterreaderB277327D": { "DeletionPolicy": "Delete", "DependsOn": [ "DbDbClusterwriterCE53952E", ], "Properties": { "DBClusterIdentifier": { "Ref": "DbDbCluster8FBC7FAD",

Now, cloudformation wants to delete and recreate the database in order to apply that change, which is obviously not convenient.

So… any ideas how we can deploy this change without deleting the DB and having to re-load the data?

Expected Behavior

It should not delete DB

Current Behavior

Stack update is failing because of DB recreation

Reproduction Steps

const dbCluster = new rds.DatabaseCluster(this, "DbCluster", {
  serverlessV2MinCapacity: 1,
  serverlessV2MaxCapacity: 10,
  writer: rds.ClusterInstance.serverlessV2("writer"),
  readers: [rds.ClusterInstance.serverlessV2("reader")],
  storageEncryptionKey: dbStorageEncryptionKmsKey,
  removalPolicy: RemovalPolicy.DESTROY,
  copyTagsToSnapshot: true,
  clusterIdentifier: dbName,
  securityGroups: [dbSecurityGroup],
  credentials: dbClusterCredentials,
  vpc,
  vpcSubnets: privateSubnets,
  defaultDatabaseName: dbDatabaseName,
  engine: rds.DatabaseClusterEngine.auroraMysql({
    version: rds.AuroraMysqlEngineVersion.VER_3_05_2,
  }),
});

Our old DB constuct , now updating

cdk versions

Possible Solution

not sure if this is causing issue : https://github.com/aws/aws-cdk/issues/30260

Additional Information/Context

No response

CDK CLI Version

aws-cdk (source) devDependencies minor 2.142.0 -> 2.143.0

Framework Version

No response

Node.js Version

20

OS

Linux

Language

TypeScript

Language Version

No response

Other information

No response

colifran commented 4 weeks ago

I think this PR is where the explicit dependency was added. @pahud any thoughts on this?

pahud commented 4 weeks ago

Yes that PR ensures the writer would always be created before the reader.

Hi @mamta925

Did you mean you created this in 2.142.0 and now you just tried to upgrade to 2.143.0 and CDK is trying to replace your nodes?

const dbCluster = new rds.DatabaseCluster(this, "DbCluster", {
  serverlessV2MinCapacity: 1,
  serverlessV2MaxCapacity: 10,
  writer: rds.ClusterInstance.serverlessV2("writer"),
  readers: [rds.ClusterInstance.serverlessV2("reader")],
  storageEncryptionKey: dbStorageEncryptionKmsKey,
  removalPolicy: RemovalPolicy.DESTROY,
  copyTagsToSnapshot: true,
  clusterIdentifier: dbName,
  securityGroups: [dbSecurityGroup],
  credentials: dbClusterCredentials,
  vpc,
  vpcSubnets: privateSubnets,
  defaultDatabaseName: dbDatabaseName,
  engine: rds.DatabaseClusterEngine.auroraMysql({
    version: rds.AuroraMysqlEngineVersion.VER_3_05_2,
  }),
});
pahud commented 4 weeks ago

trying to reproduce this in my account

pahud commented 4 weeks ago

Hi @mamta925

I created the cluster with 2.142.0 as below:

export class DummyStack extends Stack {
  constructor(scope: Construct, id: string, props?: StackProps) {
    super(scope, id, props);

    const vpc = ec2.Vpc.fromLookup(this, 'Vpc', { isDefault: true });

    const dbCluster = new rds.DatabaseCluster(this, "DbCluster", {
      serverlessV2MinCapacity: 1,
      serverlessV2MaxCapacity: 10,
      writer: rds.ClusterInstance.serverlessV2("writer"),
      readers: [rds.ClusterInstance.serverlessV2("reader")],
      // storageEncryptionKey: dbStorageEncryptionKmsKey,
      removalPolicy: RemovalPolicy.DESTROY,
      copyTagsToSnapshot: true,
      // clusterIdentifier: dbName,
      // securityGroups: [dbSecurityGroup],
      // credentials: dbClusterCredentials,
      vpc,
      // vpcSubnets: privateSubnets,
      // defaultDatabaseName: dbDatabaseName,
      engine: rds.DatabaseClusterEngine.auroraMysql({
        version: rds.AuroraMysqlEngineVersion.VER_3_05_2,
      }),
    });

  }
}

Then I upgraded to 2.143.0. The cdk diff indicates the reader node is going to have a DependOn on the writer with in-place update and it took 10 seconds on the deployment. No node is being replaced.

image

Now, cloudformation wants to delete and recreate the database in order to apply that change, which is obviously not convenient.

What made you think CFN was trying to delete and recreate the instance?

mamta925 commented 4 weeks ago

Hey, yes this part only, As soon as I added below code(removed dependency ) then every thing worked

   const  cfnDbReader = dbCluster.node.findChild("reader").node.defaultChild as CfnResource;
    cfnDbReader.addOverride("DependsOn", undefined);

Screenshot 2024-06-05 at 8 10 29 AM