Dyn provider "not acceptable" response on record creation

This issue was originally opened by @daveadams as hashicorp/terraform#10607. It was migrated here as part of the provider split. The original body of the issue is below.

Terraform Version

Terraform v0.7.13

Affected Resource(s)

dyn_record

Terraform Configuration Files

provider "dyn" {
  customer_name = "mycorp"
}

resource "dyn_record" "cname" {
  zone  = "example.com"
  name  = "myapp"
  value = "other-name.example.com."
  type  = "CNAME"
  ttl   = 300
}

Expected Behavior

The CNAME record should be created, and Terraform should recognize this and add it to the state.

Actual Behavior

About half the time, when creating a new CNAME, we get:

Error applying plan:

1 error(s) occurred:

* dyn_record.cname: Failed to find Dyn record id: Bad response, got "406 Not Acceptable"

That error message tracks back to vendor/github.com/nesv/go-dynect/dynect/convenient_client.go in the GetRecordID function, which in turn is called in builtin/providers/dyn/resource_dyn_record.go in the resourceDynRecordCreate and resourceDynRecordUpdate functions.

Steps to Reproduce

terraform apply

Important Factoids

This behavior is intermittent, and we have worked around it by deleting the record by hand and re-running Terraform until it works.

I am guessing this error is basically a race condition with the Dyn API, where the API query for the record ID comes too quickly after the commit step.

I believe the fix should be just to implement a simple retry loop with incremental delay around the GetRecordID function. I assume similar logic exists elsewhere in Terraform, but I haven't had a chance to look for something that could be reused around this code.

hashicorp / terraform-provider-dyn