aws-cdk-lib/aws-route53: deleteExisting does not work for multi-region latency record set

[antoniordz96]

Describe the bug

We are deploying a multi region API with a route53 recordSet as shown below.

The stack creates successfully in the east region but fails to provision in the west region as it seems the custom resource is attempting to delete a non-existing record.

 Reason: "InvalidChangeBatch: [Tried to delete resource record set [name='snow-dev.cool-account.aws.company.com.', type='A'] but it was not found]\n" +
    '    at Request.extractError (/var/runtime/node_modules/aws-sdk/lib/protocol/rest_xml.js:53:29)\n' +
    '    at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:106:20)\n' +
    '    at Request.emit (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:78:10)\n' +
    '    at Request.emit (/var/runtime/node_modules/aws-sdk/lib/request.js:686:14)\n' +
    '    at Request.transition (/var/runtime/node_modules/aws-sdk/lib/request.js:22:10)\n' +
    '    at AcceptorStateMachine.runTo (/var/runtime/node_modules/aws-sdk/lib/state_machine.js:14:12)\n' +
    '    at /var/runtime/node_modules/aws-sdk/lib/state_machine.js:26:10\n' +
    '    at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:38:9)\n' +
    '    at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:688:12)\n' +
    '    at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:116:18)',

We believe we have tracked this down to the following request listResourceRecordSets in the custom resource which returns the A record set in us-east-1. The custom resource proceeds to attempt delete the A record and fails.

Expected Behavior

The custom resource on creation should not attempt to delete A non existent A record in the region

Current Behavior

Stack creation in second region fails to provision due to custom resource not being able to delete A record.

Reproduction Steps

  private createHostedZoneRecord(props: DeltaPrivateApiNetworkingProps) {
    switch (this.route53RecordRoutingPolicy) {
      case RoutingPolicy.SIMPLE:
        this.createArecord(props);
        break;
      case RoutingPolicy.LATENCY:
        let aRecord = this.createArecord(props);
        // For latency Routing policy, we have to set Region and Identifier fields
        const recordSet = (aRecord.node.defaultChild as CfnRecordSet);
        recordSet.region = props.region;
        recordSet.setIdentifier = props.region;
        recordSet.healthCheckId = props.route53HealthCheckId;
        break;
      case RoutingPolicy.DISABLED:
        break;
      default:
        throw Error('Invalid Route53 Record Routing Policy!');
    }
  }

  private createArecord(props: DeltaPrivateApiNetworkingProps) {
    // Create an A type of record
    return new ARecord(this, 'ARecord', {
      target: RecordTarget.fromAlias(new LoadBalancerTarget(this.applicationLoadBalancer)),
      zone: this.route53HostedZone,
      recordName: `${props.route53RecordName}.${this.route53HostedZone.zoneName}`,
      deleteExisting: true, // TODO:: yellowstone to investigate more on this
    });;
  }

Possible Solution

No response

Additional Information/Context

No response

CDK CLI Version

2.77.0

Framework Version

No response

Node.js Version

v16.16.0

OS

Mac

Language

Typescript

Language Version

No response

Other information

No response

[peterwoodworth]

Thanks for reporting this,

Could you post some clearer reproduction steps, ideally code that I can copy+paste? It's not clear to me how exactly you have this resource configured, it sounds like you're deploying to multiple regions?

[github-actions[bot]]

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.