"Root resource was present, but now absent" adding NS to existing A record

Jackbennett commented 3 years ago

Hi there,

Adding A record, add subdomain to A record, add NS for A record then breaks.

# bind9 updates log
# tf apply just A
20-Sep-2021 16:45:11.805 update: info: client @0x7f849054a8 10.1.1.15#57184/key remote-rndc-key: view example-users: updating zone 'example.dev/IN': adding an RR at 'ci.example.dev' A 198.1.1.57
# tf apply just sub.A
20-Sep-2021 16:46:47.691 update: info: client @0x7f8c4fc838 10.1.1.15#44574/key remote-rndc-key: view example-users: updating zone 'example.dev/IN': adding an RR at 'admin.ci.example.dev' A 198.1.1.21
# tf apply NS
20-Sep-2021 16:48:39.358 update: info: client @0x7f8434ca48 10.1.1.15#35086/key remote-rndc-key: view example-users: updating zone 'example.dev/IN': adding an RR at 'ci.example.dev' NS ci.example.dev.
# tf apply, all together.
20-Sep-2021 16:55:19.811 update: info: client @0x7f8434ca48 10.1.1.15#32930/key remote-rndc-key: view example-users: updating zone 'example.dev/IN': adding an RR at 'ci.example.dev' A 198.1.1.57
20-Sep-2021 16:55:19.819 update: info: client @0x7f849054a8 10.1.1.15#57226/key remote-rndc-key: view example-users: updating zone 'example.dev/IN': adding an RR at 'admin.ci.example.dev' A 198.1.1.21

Terraform Version

Terraform v1.0.7
on linux_amd64
+ provider registry.terraform.io/hashicorp/dns v3.2.1

Affected Resource(s)

maybe dns_ns_record_set ?

Terraform Configuration Files

locals {
  exampledev_zone = "example.dev."
  public_example = toset([
    "admin.ci",
  ])
}

resource "dns_a_record_set" "public_example" {
  zone      = local.exampledev_zone
  for_each  = local.public_example
  name      = each.key
  addresses = ["10.9.8.21"]
}

resource "dns_a_record_set" "k8s" {
  zone      = local.eampledev_zone
  name      = "ci"
  addresses = ["10.9.8.57"]
}

# resource "dns_ns_record_set" "kube_root" {
#   zone        = local.exampledev_zone
#   name        = "ci"
#   nameservers = [dns_a_record_set.k8s.id]
# }

tf apply: Apply complete! Resources: 2 added, 0 changed, 0 destroyed.
tf plan: No changes. Your infrastructure matches the configuration.

and the applied zonefile, no NS record:

$ORIGIN example.dev.
$TTL 3600       ; 1 hour
ci                      A       198.1.1.57
$ORIGIN ci.example.dev.
admin                   A       198.1.1.21
$ORIGIN example.dev.
...

Add NS

resource "dns_ns_record_set" "kube_root" {
  zone        = local.exampledev_zone
  name        = "ci"
  nameservers = [dns_a_record_set.k8s.id]
}

tf plan:

Terraform will perform the following actions:

  # dns_ns_record_set.kube_root will be created
  + resource "dns_ns_record_set" "kube_root" {
      + id          = (known after apply)
      + name        = "ci"
      + nameservers = [
          + "ci.example.dev.",
        ]
      + ttl         = 3600
      + zone        = "example.dev."
    }

Plan: 1 to add, 0 to change, 0 to destroy.

tf apply: Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

tf plan:

Note: Objects have changed outside of Terraform

Terraform detected the following changes made outside of Terraform since the last "terraform apply":

  # dns_a_record_set.k8s has been deleted
  - resource "dns_a_record_set" "k8s" {
      - addresses = [
          - "198.1.1.57",
        ] -> null
      - id        = "ci.example.dev." -> null
      - name      = "ci" -> null
      - ttl       = 3600 -> null
      - zone      = "example.dev." -> null
    }
  # dns_a_record_set.public_example["admin.ci"] has been deleted
  - resource "dns_a_record_set" "public_example" {
      - addresses = [
          - "198.1.1.21",
        ] -> null
      - id        = "admin.ci.example.dev." -> null
      - name      = "admin.ci" -> null
      - ttl       = 3600 -> null
      - zone      = "example.dev." -> null
    }

Unless you have made equivalent changes to your configuration, or ignored the relevant attributes using ignore_changes, the following plan may include actions to undo or respond to these changes.

───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create
  ~ update in-place

Terraform will perform the following actions:

  # dns_a_record_set.k8s will be created
  + resource "dns_a_record_set" "k8s" {
      + addresses = [
          + "198.1.1.57",
        ]
      + id        = (known after apply)
      + name      = "ci"
      + ttl       = 3600
      + zone      = "example.dev."
    }

  # dns_a_record_set.public_example["admin.ci"] will be created
  + resource "dns_a_record_set" "public_example" {
      + addresses = [
          + "198.1.1.21",
        ]
      + id        = (known after apply)
      + name      = "admin.ci"
      + ttl       = 3600
      + zone      = "example.dev."
    }

  # dns_ns_record_set.kube_root will be updated in-place
  ~ resource "dns_ns_record_set" "kube_root" {
        id          = "ci.example.dev."
        name        = "ci"
      ~ nameservers = [
          - "ci.example.dev.",
        ] -> (known after apply)
        # (2 unchanged attributes hidden)
    }

Plan: 2 to add, 1 to change, 0 to destroy.

zone after

$ORIGIN example.dev.
$TTL 3600       ; 1 hour
ci                      NS      ci
                        A       198.1.1.57
$ORIGIN ci.example.dev.
admin                   A       198.1.1.21
$ORIGIN example.dev.
...

tf apply:

dns_a_record_set.k8s: Creating...
dns_a_record_set.public_example["admin.ci"]: Creating...
╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to dns_a_record_set.public_example["admin.ci"], provider "provider[\"registry.terraform.io/hashicorp/dns\"]" produced an unexpected new value: Root resource was present, but now
│ absent.
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.
╵
╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to dns_a_record_set.k8s, provider "provider[\"registry.terraform.io/hashicorp/dns\"]" produced an unexpected new value: Root resource was present, but now absent.
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.

Debug Output

any tips before I spend 30min sanitizing the debug log?

Expected Behavior

Created dns A record, subdomain in A record and also NS for A record

Actual Behavior

Records Created, but terraform chokes on subsequent runs.

Important Factoids

TF from ubuntu linux targeting an ubuntu bind9 server. no clouds involved.

References

Are there any other GitHub issues (open or closed) or Pull Requests that should be linked here? For example:

could https://github.com/hashicorp/terraform-provider-dns/issues/148 be related?

Sorry to adjust your issue template slightly, I had most of this formatted while debugging the issue myself. Please forgive me.

macdweller commented 3 years ago

I am seeing almost identical behavior in a very similar situation.

My nameserver hosts domain parent.io, and also subdomain child.parent.io; if I try to update glue NS records within parent.io (which should appear in parent.io's zone file), I get the above issue. Terraform import can also not import glue NS records in the parent zone.

This is for something defined in the style of:

resource "dns_ns_record_set" "parent-glue" {
  zone        = "parent.io."
  name        = "child"
  nameservers = [ "ns1.parent.io.", "ns2.parent.io." ]
  ttl         = 3600 
}

Note that this only applies on a server that is ALSO hosting the child domain (or perhaps that the child domain just plain exists? I haven't tested it with an extant domain primarily hosted on a different server). If the child domain does not exist at the server, it will happily add/modify/destroy the glue NS records for it in the parent zone.

My best guess is that it's adding the record to the correct zone, but then reading data from the child zone rather than the parent when it goes to verify it. That's only a guess, however.

pbnsh commented 2 years ago

Having exact same issue when trying to delegate a subdomain, initial run is fine, subsequent ones are broken. terraform 1.1.6, terraform-provider-dns 3.2.1

Jackbennett commented 2 years ago

My best guess is that it's adding the record to the correct zone, but then reading data from the child zone rather than the parent when it goes to verify it. That's only a guess, however.

My workaround to this has been to create an actual zone for ci.example.com that only has the nameservers and the single admin A record so terraform behaves on subsequent runs. So I think as quoted has nailed the issue cause.

richard-sistern commented 2 years ago

Ran into this when trying to create an entry with _dns_a_recordset.

Version

Terraform v1.1.8 on windows_amd64

provider registry.terraform.io/hashicorp/dns v3.2.3

Example code

resource "dns_a_record_set" "example" {
  zone = "zone.com."
  name = "one.two.three"
  addresses = [
    "8.8.8.8"
  ]
}

Workaround

Add a stub.two.three.zone.com entry manually
Run terrform apply
Remove stub

Jackbennett commented 2 years ago

Workaround

Add a stub.two.three.zone.com entry manually

Run terrform apply

Remove stub

Does that behave upon multiple re-apply runs after? That's what I found the real bug from, you can get it to run but the issues come from running again.

richard-sistern commented 2 years ago

So far, it's been fine on multiple applies. Seems to require the subdomain to be present before creating records there.

PowerSurj commented 2 years ago

So far, it's been fine on multiple applies. Seems to require the subdomain to be present before creating records there.

This is did not work for me for NS resource records on MS DNS (2019) Tested using DNS provider 3.2.3 and it still doesn't work. Delegation is not being created. DNS Server responds with REFUSED

See terraform trace below:

2022-12-02T02:06:45.561Z [INFO]  provider.terraform-provider-dns_v3.2.3_x5: 2022/12/02 02:06:45 [DEBUG] Sending DNS message to server (host.mydomain.com:53):
;; opcode: UPDATE, status: NOERROR, id: 18833
;; flags:; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; QUESTION SECTION:
;domain.com.    IN   SOA

;; AUTHORITY SECTION:
sub.domain.com. 60  IN  NS  ns9-09.azure-dns.com.

;; ADDITIONAL SECTION:

;; TSIG PSEUDOSECTION:
; 484559840.sig-host.mydomain.com.  0   CLASS255    TSIG     gss-tsig. 20221202020645 300 0  18833 0 0: timestamp=2022-12-02T02:06:45.561Z
2022-12-02T02:06:45.566Z [INFO]  provider.terraform-provider-dns_v3.2.3_x5: 2022/12/02 02:06:45 [DEBUG] Receiving DNS message from server (host.mydomain.com:53):
;; opcode: UPDATE, status: REFUSED, id: 18833
;; flags: qr; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; QUESTION SECTION:
;domain.com.    IN   SOA

;; AUTHORITY SECTION:
sub.domain.com. 60  IN  NS  ns9-09.azure-dns.com.

;; ADDITIONAL SECTION:

;; TSIG PSEUDOSECTION:

Enabled DNS server audit logs and they don't record anything meaningful. No other Windows server logs indicate the reason. Any suggestions?

PowerSurj commented 1 year ago

I'd appreciate someone's suggestions/findings in troubleshooting this on the Windows server side. We could tackle this problem from this size.

bflad commented 1 year ago

The Terraform Provider produced inconsistent result after apply error with the details of Root resource was present, but now absent. should be removed by the changes in #260, however it may be the case that there is still another underlying issue which either needs to be fixed or documented better, so I'm going to leave this issue open until we see what the version 3.3.0 (releasing in the near future) behavior is after those changes.

hashicorp / terraform-provider-dns