AWS::RDS::DBInstance fails to update certain settings if in a VPC shared via RAM

bracki commented 4 years ago

1. Title

AWS::RDS::DBInstance fails to update certain settings if in a VPC shared via RAM

2. Scope of request

Make RDS and Cloudformation work in a shared VPC. With VPC sharing via RAM Cloudformation can't be relied on when using RDS.

3. Expected behavior

Create a VPC in one account and share via RAM
Create an Aurora cluster with one instance in the shared VPC in another account.
Update the instance's DBInstanceClass or EnablePerformanceMetrics (there are probably more, but these are the two I ran into)
Stack update should complete successfully instead of events with InternalFailure

Trying to work around these limitations by removing the instance from the stack, updating it by hand and then reimporting also fails with InternalFailure.

4. Suggest specific test cases

Samples:

Create a VPC in one account and share via RAM
Create an Aurora cluster with one instance in the shared VPC in another account.
Update the instance's DBInstanceClass or EnablePerformanceMetrics (there are probably more, but these are the two I ran into)
Stack update should complete successfully instead of events with InternalFailure

5. Helpful Links to speed up research and evaluation

The support mentioned that the team already knows about this.

6. Category (required) - Will help with tagging and be easier to find by other users to +1

DB (RDS, DynamoDB...)
Networking & Content (VPC, Route53, API GW,...)

robertd commented 4 years ago

@bracki @luiseduardocolon This bug is also preventing us from moving RDS instances to shared VPC (owned by different account), thus preventing us from moving forward with cross VPC migration of all of our AWS resources. Any update on this would be greatly appreciated. We also have AWS Support Case ID tracking the issue: 6844571841.

jls-tschanzc commented 4 years ago

I face a similar issue (CDK Repo: #6439 and #6542):

Creating an RDS DatabaseInstance through CDK and later changing various properties (e.g. "engineVersion") results in an "Internal Failure" error message. In my case I did not import the VPC from another account, I created everything in the same stack:

interface BackendStackProps extends StackProps {
  deployStage?: string;
}

export class BackendStack extends Stack {
  constructor(scope: Construct, id: string, props?: BackendStackProps) {
    super(scope, id, props);

    // --- Networking
    const vpc = new Vpc(this, 'TestVPC', {
      natGateways: 1,
      maxAzs: 2,
    });

    // --- Database
    const appDB = new DatabaseInstance(this, 'TestPostgresInstance', {
      engine: DatabaseInstanceEngine.POSTGRES,
      instanceClass: InstanceType.of(InstanceClass.BURSTABLE3, InstanceSize.MICRO),
      masterUsername: 'TestUser',
      databaseName: 'testdb',
      vpc,
      allocatedStorage: 10,
      backupRetention: Duration.days(3),
    });

    // --- Grant Service Permissions
    // "service" is an instance of "ApplicationLoadBalancedFargateService"
    // appDB.connections.allowDefaultPortFrom(service.service.connections);
    // appDB.secret?.grantRead(service.taskDefinition.taskRole);
    // const secretPolicy = new Policy(this, 'DBSecretPolicy', {
    //   statements: [
    //     new PolicyStatement({
    //       effect: Effect.ALLOW,
    //       resources: [appDB.secret?.secretArn || ''],
    //       actions: ['secretsmanager:DescribeSecret'],
    //     }),
    //   ],
    // });
    // service.taskDefinition.taskRole.attachInlinePolicy(secretPolicy); // See: https://github.com/aws/aws-cdk/issues/6444
  }
}

nikoizs commented 4 years ago

Is there an update on the fix? it's in "Coming Soon" for 5 weeks already. We are using shared VPCs and we can't update our RDS databases with Cloudformation. Any update will be appreciated.

robertd commented 4 years ago

@luiseduardocolon Any updates on this would be much appreciated. Thank you!

robertd commented 4 years ago

Hello,

Any updates on this?

robertd commented 3 years ago

Hello Robert,

This is Eric again, and I wanted to provide you with an update.

Although the issue has not been updated in our public roadmap on GitHub, our internal CloudFormation team has verified that a fix has been implemented from our end to mitigate the issue, and you should no longer be encountering the same error.

Please let me know if you have any additional questions or concerns, and I will be happy to further assist you.

We value your feedback. Please share your experience by rating this correspondence using the AWS Support Center link at the end of this correspondence. Each correspondence can also be rated by selecting the stars in top right corner of each correspondence within the AWS Support Center.

Best regards, Eric H. Amazon Web Services

mday-ntirety commented 1 year ago

I think we are encountering this issue also. We are using a VPC, but that VPC is not shared. I think this applies to any custom VPC, not just shared ones.

When I try to update VPCSecurityGroups property or even MonitoringInterval property I get the follow error from cloudformation:

Resource handler returned message: "null" (RequestToken: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx, HandlerErrorCode: InternalFailure)

There is nothing in cloudtrail that would indicate why this is happening. The failure does not seem to get published to cloudtrail at all.

moataz-mhmd commented 10 months ago

Hey @mday-ntirety, is the issue still happening? If it is, could you please provide the exact steps you took and the CFN stack templates you used to reproduce it?

aws-cloudformation / cloudformation-coverage-roadmap