Reprovisioning child of composite Route 53 healthcheck fails

[ghost]

This issue was originally opened by @jessebrennan as hashicorp/terraform#22171. It was migrated here as a result of the provider split. The original body of the issue is below.

---

Terraform Version

Terraform v0.12.5
+ provider.aws v2.20.0
+ provider.google v2.11.0
+ provider.null v2.1.2

Terraform Configuration Files

{
    "resource": [
        {
            "aws_route53_health_check": {
                "indexer": {
                    "fqdn": "...",
                    "port": 443,
                    "type": "HTTPS",
                    "resource_path": "/health",
                    "failure_threshold": "3",
                    "request_interval": "30",
                    "tags": {
                        "Name": "..."
                    },
                    "measure_latency": false,
                }
            }
        },
        {
            "aws_route53_health_check": {
                "service": {
                    "fqdn": "...",
                    "port": 443,
                    "type": "HTTPS",
                    "resource_path": "/health",
                    "failure_threshold": "3",
                    "request_interval": "30",
                    "tags": {
                        "Name": "..."
                    },
                    "measure_latency": false,
                }
            }
        },
        {
            "aws_route53_health_check": {
                "composite-test": {
                    "reference_name": "composite-test",
                    "type": "CALCULATED",
                    "child_health_threshold": 2,
                    "child_healthchecks": [
                        "${aws_route53_health_check.indexer.id}",
                        "${aws_route53_health_check.service.id}"
                    ],
                    "cloudwatch_alarm_region": "us-east-1",
                    "tags": {
                        "Name": "..."
                    }
                }
            }
        }
   ]
}

Debug Output

Crash Output

Expected Behavior

Terraform should successfully provision updates to children of composite health checks.

Actual Behavior

Under the right conditions these errors will occur.

aws_cloudwatch_log_group.service: Creating...
aws_route53_health_check.indexer: Destroying... [id=ae09fce1-3ebc-4390-a5a8-598604166027]
aws_route53_health_check.service: Destroying... [id=c59e4195-8030-4c04-a7de-dabcb1499806]
aws_cloudwatch_log_group.indexer: Creating...

Error: InvalidInput: Invalid parameter : Health check ae09fce1-3ebc-4390-a5a8-598604166027 is still referenced from parent health check(s): d719550f-8081-48e7-b79b-c13d862bdf3d
        status code: 400, request id: cb702543-c3fa-494f-a7a6-b1e53880ede6

Error: InvalidInput: Invalid parameter : Health check c59e4195-8030-4c04-a7de-dabcb1499806 is still referenced from parent health check(s): d719550f-8081-48e7-b79b-c13d862bdf3d
        status code: 400, request id: 1915d0cc-dc6e-48d5-a09d-09d1ce73b8d6

Steps to Reproduce

1. terraform init
2. terraform apply
3. Change a field of the health check that forces re-provisioning. For example set measure_latency to true.
4. terraform apply

Additional Context

This can be worked around by adding

                    "lifecycle": {
                        "create_before_destroy": true
                    }

to the child health checks.

References

[whizz0]

I have a similar issue, that is based on the same problem. I have multiple child R53 healthchecks created by TF and then another resource as "CALCULATED" R53 healtcheck, where I pass the IDs of all the childs.

AWS provider version: 3.9.0

It works fine when creating (childs are getting created first, then the parent which uses their IDs = all good), but it fails when I want to remove one or more of the childs (TF tries to destroy the child first, but it's still referenced in the parent):

resource "aws_route53_health_check" "child" {
  for_each = var.r53_healtchecks_enabled == true ? toset(var.services) : []

  failure_threshold = var.healthcheck_failure_treshold
  fqdn              = var.domain_name
  port              = var.healthcheck_port
  request_interval  = "30"
  resource_path     = "/${each.key}${var.healthcheck_path}"
  search_string     = var.healthcheck_expected_value
  type              = "HTTPS_STR_MATCH"
}

resource "aws_route53_health_check" "parent" {
  count = var.r53_healtchecks_enabled == true ? 1 : 0

  type                   = "CALCULATED"
  child_health_threshold = floor(var.child_health_threshold_percentage / 100 * length(var.services))
  child_healthchecks     = values(aws_route53_health_check.child)[*].id
}

---

After removing some values from the list var.services the error occurs (provider is trying to destroy the childs, before unplugging them from the parent):

module.route53_healthcheck.aws_route53_health_check.child["service_1"]: Destroying... [id=a7035239-0cdb-4ad7-a8e1-9470e1ca3d10]
module.route53_healthcheck.aws_route53_health_check.child["service_2"]: Destroying... [id=dcd4c4cb-6c70-4a3e-a54a-574d66391c04]

Error: InvalidInput: Invalid parameter : Health check dcd4c4cb-6c70-4a3e-a54a-574d66391c04 is still referenced from parent health check(s): adc2b973-5469-4a0c-a337-db467020dd8b
    status code: 400, request id: 196c409a-c4ce-4e46-bec3-9ba1d0e184a9

Error: InvalidInput: Invalid parameter : Health check a7035239-0cdb-4ad7-a8e1-9470e1ca3d10 is still referenced from parent health check(s): adc2b973-5469-4a0c-a337-db467020dd8b
    status code: 400, request id: cd9620b9-37dc-433d-b3c8-bce7933a85cc

The "create_before_destroy" workaround proposed is not working in the case when I just want to remove a couple of child healtchecks...

[MicahRam]

Same issue with provider 3.46.0

[julian-alarcon]

Same issue with provider version v4.67.0

aws_route53_health_check.cloudfront_total_errors_old_api_flixbus_com: Destroying... [id=ID_HEALTHCHECK01]
aws_route53_health_check.cloudfront_total_errors_global_api_flixbus_com: Destroying... [id=ID_HEALTHCHECK02]
╷
│ Error: deleting Route53 Health Check (ID_HEALTHCHECK01): InvalidInput: Invalid parameter : Health check ID_HEALTHCHECK01 is still referenced from parent health check(s): ID_PARENT_HEALTHCHECK01
│       status code: 400, request id: REQUEST_ID_01
│ 
│ Error: deleting Route53 Health Check (ID_HEALTHCHECK02): InvalidInput: Invalid parameter : Health check ID_HEALTHCHECK02 is still referenced from parent health check(s): ID_PARENT_HEALTHCHECK02
│       status code: 400, request id: REQUEST_ID_02