cloudflare / terraform-provider-cloudflare

Cloudflare Terraform Provider
https://registry.terraform.io/providers/cloudflare/cloudflare
Mozilla Public License 2.0
754 stars 576 forks source link

cloudflare_custom_hostname does not keep showing ownership_verification or validation_records after verification #1466

Open bdandoy opened 2 years ago

bdandoy commented 2 years ago

Confirmation

Terraform and Cloudflare provider version

Terraform v1.0.11 on darwin_arm64

provider registry.terraform.io/cloudflare/cloudflare v3.9.1 provider registry.terraform.io/hashicorp/aws v3.74.2

Affected resource(s)

Terraform configuration files

resource "cloudflare_custom_hostname" "sans" {
  zone_id  = "6a9006164edb31aca836cf6f4d0c6184"
  hostname = "custom.paradox.ai"
  ssl {
    method = "txt"
    settings {
      http2 = "on"
      tls13 = "on"
      min_tls_version = "1.2"
    }
  }
}

Debug output

2022-02-17T08:07:02.607-0700 [DEBUG] Adding temp file log sink: /var/folders/hj/sd6xmf7x62798sq7d6z6kp040000gn/T/terraform-log201296396 2022-02-17T08:07:02.607-0700 [INFO] Terraform version: 1.0.11 2022-02-17T08:07:02.607-0700 [INFO] Go runtime version: go1.16.4 2022-02-17T08:07:02.607-0700 [INFO] CLI args: []string{"/Users/brian.dandoy/.asdf/installs/terraform/1.0.11/bin/terraform", "apply"} 2022-02-17T08:07:02.607-0700 [DEBUG] Attempting to open CLI config file: /Users/brian.dandoy/.terraformrc 2022-02-17T08:07:02.607-0700 [DEBUG] File doesn't exist, but doesn't need to. Ignoring. 2022-02-17T08:07:02.607-0700 [DEBUG] ignoring non-existing provider search directory terraform.d/plugins 2022-02-17T08:07:02.607-0700 [DEBUG] ignoring non-existing provider search directory /Users/brian.dandoy/.terraform.d/plugins 2022-02-17T08:07:02.607-0700 [DEBUG] ignoring non-existing provider search directory /Users/brian.dandoy/Library/Application Support/io.terraform/plugins 2022-02-17T08:07:02.607-0700 [DEBUG] ignoring non-existing provider search directory /Library/Application Support/io.terraform/plugins 2022-02-17T08:07:02.607-0700 [INFO] CLI command args: []string{"apply"} 2022-02-17T08:07:02.608-0700 [DEBUG] New state was assigned lineage "d530782b-f7e8-fd4e-eae8-931ab1a62901" 2022-02-17T08:07:02.618-0700 [DEBUG] checking for provisioner in "." 2022-02-17T08:07:02.618-0700 [DEBUG] checking for provisioner in "/Users/brian.dandoy/.asdf/installs/terraform/1.0.11/bin" 2022-02-17T08:07:02.619-0700 [INFO] Failed to read plugin lock file .terraform/plugins/darwin_arm64/lock.json: open .terraform/plugins/darwin_arm64/lock.json: no such file or directory 2022-02-17T08:07:02.619-0700 [INFO] backend/local: starting Apply operation 2022-02-17T08:07:02.620-0700 [DEBUG] created provider logger: level=debug 2022-02-17T08:07:02.620-0700 [INFO] provider: configuring client automatic mTLS 2022-02-17T08:07:02.642-0700 [DEBUG] provider: starting plugin: path=.terraform/providers/registry.terraform.io/cloudflare/cloudflare/3.9.1/darwin_arm64/terraform-provider-cloudflare_v3.9.1 args=[.terraform/providers/registry.terraform.io/cloudflare/cloudflare/3.9.1/darwin_arm64/terraform-provider-cloudflare_v3.9.1] 2022-02-17T08:07:02.644-0700 [DEBUG] provider: plugin started: path=.terraform/providers/registry.terraform.io/cloudflare/cloudflare/3.9.1/darwin_arm64/terraform-provider-cloudflare_v3.9.1 pid=17820 2022-02-17T08:07:02.644-0700 [DEBUG] provider: waiting for RPC address: path=.terraform/providers/registry.terraform.io/cloudflare/cloudflare/3.9.1/darwin_arm64/terraform-provider-cloudflare_v3.9.1 2022-02-17T08:07:02.651-0700 [INFO] provider.terraform-provider-cloudflare_v3.9.1: configuring server automatic mTLS: timestamp=2022-02-17T08:07:02.651-0700 2022-02-17T08:07:02.658-0700 [DEBUG] provider: using plugin: version=5 2022-02-17T08:07:02.658-0700 [DEBUG] provider.terraform-provider-cloudflare_v3.9.1: plugin address: address=/var/folders/hj/sd6xmf7x62798sq7d6z6kp040000gn/T/plugin1238446969 network=unix timestamp=2022-02-17T08:07:02.658-0700 2022-02-17T08:07:02.684-0700 [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = transport is closing" 2022-02-17T08:07:02.684-0700 [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/cloudflare/cloudflare/3.9.1/darwin_arm64/terraform-provider-cloudflare_v3.9.1 pid=17820 2022-02-17T08:07:02.684-0700 [DEBUG] provider: plugin exited 2022-02-17T08:07:02.685-0700 [INFO] terraform: building graph: GraphTypeValidate 2022-02-17T08:07:02.685-0700 [DEBUG] ProviderTransformer: "cloudflare_custom_hostname.sans" (terraform.NodeValidatableResource) needs provider["registry.terraform.io/cloudflare/cloudflare"] 2022-02-17T08:07:02.685-0700 [DEBUG] ReferenceTransformer: "cloudflare_custom_hostname.sans" references: [] 2022-02-17T08:07:02.685-0700 [DEBUG] ReferenceTransformer: "provider[\"registry.terraform.io/cloudflare/cloudflare\"]" references: [] 2022-02-17T08:07:02.685-0700 [DEBUG] Starting graph walk: walkValidate 2022-02-17T08:07:02.686-0700 [DEBUG] created provider logger: level=debug 2022-02-17T08:07:02.686-0700 [INFO] provider: configuring client automatic mTLS 2022-02-17T08:07:02.707-0700 [DEBUG] provider: starting plugin: path=.terraform/providers/registry.terraform.io/cloudflare/cloudflare/3.9.1/darwin_arm64/terraform-provider-cloudflare_v3.9.1 args=[.terraform/providers/registry.terraform.io/cloudflare/cloudflare/3.9.1/darwin_arm64/terraform-provider-cloudflare_v3.9.1] 2022-02-17T08:07:02.709-0700 [DEBUG] provider: plugin started: path=.terraform/providers/registry.terraform.io/cloudflare/cloudflare/3.9.1/darwin_arm64/terraform-provider-cloudflare_v3.9.1 pid=17821 2022-02-17T08:07:02.709-0700 [DEBUG] provider: waiting for RPC address: path=.terraform/providers/registry.terraform.io/cloudflare/cloudflare/3.9.1/darwin_arm64/terraform-provider-cloudflare_v3.9.1 2022-02-17T08:07:02.716-0700 [INFO] provider.terraform-provider-cloudflare_v3.9.1: configuring server automatic mTLS: timestamp=2022-02-17T08:07:02.715-0700 2022-02-17T08:07:02.723-0700 [DEBUG] provider: using plugin: version=5 2022-02-17T08:07:02.723-0700 [DEBUG] provider.terraform-provider-cloudflare_v3.9.1: plugin address: network=unix address=/var/folders/hj/sd6xmf7x62798sq7d6z6kp040000gn/T/plugin974722975 timestamp=2022-02-17T08:07:02.723-0700 2022-02-17T08:07:02.747-0700 [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = transport is closing" 2022-02-17T08:07:02.748-0700 [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/cloudflare/cloudflare/3.9.1/darwin_arm64/terraform-provider-cloudflare_v3.9.1 pid=17821 2022-02-17T08:07:02.748-0700 [DEBUG] provider: plugin exited 2022-02-17T08:07:02.748-0700 [INFO] backend/local: apply calling Plan 2022-02-17T08:07:02.748-0700 [INFO] terraform: building graph: GraphTypePlan 2022-02-17T08:07:02.748-0700 [DEBUG] ProviderTransformer: "cloudflare_custom_hostname.sans (expand)" (terraform.nodeExpandPlannableResource) needs provider["registry.terraform.io/cloudflare/cloudflare"] 2022-02-17T08:07:02.748-0700 [DEBUG] ReferenceTransformer: "provider[\"registry.terraform.io/cloudflare/cloudflare\"]" references: [] 2022-02-17T08:07:02.748-0700 [DEBUG] ReferenceTransformer: "cloudflare_custom_hostname.sans (expand)" references: [] 2022-02-17T08:07:02.749-0700 [DEBUG] Starting graph walk: walkPlan 2022-02-17T08:07:02.749-0700 [DEBUG] created provider logger: level=debug 2022-02-17T08:07:02.749-0700 [INFO] provider: configuring client automatic mTLS 2022-02-17T08:07:02.771-0700 [DEBUG] provider: starting plugin: path=.terraform/providers/registry.terraform.io/cloudflare/cloudflare/3.9.1/darwin_arm64/terraform-provider-cloudflare_v3.9.1 args=[.terraform/providers/registry.terraform.io/cloudflare/cloudflare/3.9.1/darwin_arm64/terraform-provider-cloudflare_v3.9.1] 2022-02-17T08:07:02.772-0700 [DEBUG] provider: plugin started: path=.terraform/providers/registry.terraform.io/cloudflare/cloudflare/3.9.1/darwin_arm64/terraform-provider-cloudflare_v3.9.1 pid=17822 2022-02-17T08:07:02.773-0700 [DEBUG] provider: waiting for RPC address: path=.terraform/providers/registry.terraform.io/cloudflare/cloudflare/3.9.1/darwin_arm64/terraform-provider-cloudflare_v3.9.1 2022-02-17T08:07:02.779-0700 [INFO] provider.terraform-provider-cloudflare_v3.9.1: configuring server automatic mTLS: timestamp=2022-02-17T08:07:02.779-0700 2022-02-17T08:07:02.786-0700 [DEBUG] provider: using plugin: version=5 2022-02-17T08:07:02.786-0700 [DEBUG] provider.terraform-provider-cloudflare_v3.9.1: plugin address: address=/var/folders/hj/sd6xmf7x62798sq7d6z6kp040000gn/T/plugin66265373 network=unix timestamp=2022-02-17T08:07:02.786-0700 2022-02-17T08:07:02.815-0700 [WARN] ValidateProviderConfig from "provider[\"registry.terraform.io/cloudflare/cloudflare\"]" changed the config value, but that value is unused 2022-02-17T08:07:02.815-0700 [INFO] provider.terraform-provider-cloudflare_v3.9.1: 2022/02/17 08:07:02 [INFO] Cloudflare Client configured for user: brian.dandoy@paradox.ai: timestamp=2022-02-17T08:07:02.815-0700 2022-02-17T08:07:02.815-0700 [INFO] provider.terraform-provider-cloudflare_v3.9.1: 2022/02/17 08:07:02 [INFO] Using specified account id 15e289746610ed2683be104607b16e09 in Cloudflare provider: timestamp=2022-02-17T08:07:02.815-0700 2022-02-17T08:07:02.815-0700 [INFO] provider.terraform-provider-cloudflare_v3.9.1: 2022/02/17 08:07:02 [INFO] Cloudflare Client configured for user: brian.dandoy@paradox.ai: timestamp=2022-02-17T08:07:02.815-0700 2022-02-17T08:07:02.815-0700 [DEBUG] ReferenceTransformer: "cloudflare_custom_hostname.sans" references: [] cloudflare_custom_hostname.sans: Refreshing state... [id=8d659fea-4024-4fc1-be35-f0967f898785] 2022-02-17T08:07:02.817-0700 [INFO] provider.terraform-provider-cloudflare_v3.9.1: 2022/02/17 08:07:02 [DEBUG] Cloudflare API Request Details: ---[ REQUEST ]--------------------------------------- GET /client/v4/zones/6a9006164edb31aca836cf6f4d0c6184/custom_hostnames/8d659fea-4024-4fc1-be35-f0967f898785 HTTP/1.1 Host: api.cloudflare.com User-Agent: terraform/1.0.11 terraform-plugin-sdk/2.10.1 terraform-provider-cloudflare/3.9.1 Content-Type: application/json Accept-Encoding: gzip

-----------------------------------------------------: timestamp=2022-02-17T08:07:02.817-0700 2022-02-17T08:07:03.194-0700 [INFO] provider.terraform-provider-cloudflare_v3.9.1: 2022/02/17 08:07:03 [DEBUG] Cloudflare API Response Details: ---[ RESPONSE ]-------------------------------------- HTTP/2.0 200 OK Cf-Cache-Status: DYNAMIC Cf-Ray: 6defec512cde5331-LAX Content-Type: application/json; charset=UTF-8 Date: Thu, 17 Feb 2022 15:07:03 GMT Expect-Ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct" Server: cloudflare Set-Cookie: cflb=0H28vgHxwvgAQtjUGU4vq74ZFe3sNVUZVUf2Da2uDrm; SameSite=Lax; path=/; expires=Thu, 17-Feb-22 17:37:04 GMT; HttpOnly Set-Cookie: cfruid=2e220f03cd3eb6e0e8d6bb1b9573c66d196bc752-1645110423; path=/; domain=.api.cloudflare.com; HttpOnly; Secure; SameSite=None Vary: Accept-Encoding X-Envoy-Upstream-Service-Time: 19

{ "result": { "id": "8d659fea-4024-4fc1-be35-f0967f898785", "hostname": "custom.paradox.ai", "ssl": { "id": "a4acf290-8642-4a6c-815b-3f9b223dbbd1", "type": "dv", "method": "txt", "status": "active", "hosts": [ "custom.paradox.ai" ], "settings": { "http2": "on", "tls_1_3": "on", "min_tls_version": "1.2" }, "bundle_method": "ubiquitous", "certificates": [ { "issuer": "CloudflareInc", "serial_number": "9788114406410918930987082525256964946", "signature": "ECDSAWithSHA256", "expires_on": "2023-02-16T23:59:59Z", "issued_on": "2022-02-17T00:00:00Z", "fingerprint_sha256": "9e266b7aa193c479fd7df08dd3a847f9f4aac80611b9ece39b533f34b1fbaf20", "id": "b5dbe62c-3bb8-494e-ac6d-1ba16a5ea240" }, { "issuer": "CloudflareInc", "serial_number": "10785380436403442802533380960677006285", "signature": "SHA256WithRSA", "expires_on": "2023-02-16T23:59:59Z", "issued_on": "2022-02-17T00:00:00Z", "fingerprint_sha256": "374f9c90720bb8a51ca8ba71f9fc04f1dcc2ac6ec3c3112cde2498b8d9b97ffe", "id": "d8212a89-e081-4b59-8fd4-41b568667011" } ], "wildcard": false, "certificate_authority": "digicert" }, "status": "active", "created_at": "2022-02-17T14:54:45.659754Z" }, "success": true, "errors": [], "messages": [] }

-----------------------------------------------------: timestamp=2022-02-17T08:07:03.194-0700 2022-02-17T08:07:03.198-0700 [WARN] Provider "registry.terraform.io/cloudflare/cloudflare" produced an invalid plan for cloudflare_custom_hostname.sans, but we are tolerating it because it is using the legacy plugin SDK. The following problems may be the cause of any confusing errors from downstream operations:

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed. 2022-02-17T08:07:03.201-0700 [INFO] backend/local: apply calling Apply 2022-02-17T08:07:03.201-0700 [INFO] terraform: building graph: GraphTypeApply 2022-02-17T08:07:03.201-0700 [DEBUG] ProviderTransformer: "cloudflare_custom_hostname.sans (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/cloudflare/cloudflare"] 2022-02-17T08:07:03.202-0700 [DEBUG] ReferenceTransformer: "cloudflare_custom_hostname.sans (expand)" references: [] 2022-02-17T08:07:03.202-0700 [DEBUG] ReferenceTransformer: "provider[\"registry.terraform.io/cloudflare/cloudflare\"]" references: [] 2022-02-17T08:07:03.202-0700 [DEBUG] pruneUnusedNodes: cloudflare_custom_hostname.sans (expand) is no longer needed, removing 2022-02-17T08:07:03.202-0700 [DEBUG] pruneUnusedNodes: provider["registry.terraform.io/cloudflare/cloudflare"] is no longer needed, removing 2022-02-17T08:07:03.202-0700 [DEBUG] Starting graph walk: walkApply

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Panic output

No response

Expected output

resource "cloudflare_custom_hostname" "sans" {
    hostname                    = "custom.paradox.ai"
    id                          = "8d659fea-4024-4fc1-be35-f0967f898785"
    ownership_verification      = {
        "name"  = "_cf-custom-hostname.custom.paradox.ai"
        "type"  = "txt"
        "value" = "e300d99b-4d8d-4857-ae35-ae643ad4e207"
    }
    ownership_verification_http = {
        "http_body" = "e300d99b-4d8d-4857-ae35-ae643ad4e207"
        "http_url"  = "http://custom.paradox.ai/.well-known/cf-custom-hostname-challenge/8d659fea-4024-4fc1-be35-f0967f898785"
    }
    zone_id                     = "6a9006164edb31aca836cf6f4d0c6184"
    ssl {
        certificate_authority = "digicert"
        method                = "txt"
        status                = "pending_validation"
        type                  = "dv"
        validation_errors     = []
        validation_records    = [
            {
                cname_name   = ""
                cname_target = ""
                emails       = []
                http_body    = ""
                http_url     = ""
                txt_name     = "custom.paradox.ai"
                txt_value    = "ca3-db23df47239e4fb2a077d1e386578cfa"
            },
        ]
        wildcard              = false
        settings {
            ciphers         = []
            http2           = "on"
            min_tls_version = "1.2"
            tls13           = "on"
        }
    }
}

Actual output

resource "cloudflare_custom_hostname" "sans" {
    hostname                    = "custom.paradox.ai"
    id                          = "8d659fea-4024-4fc1-be35-f0967f898785"
    ownership_verification      = {
        "name"  = ""
        "type"  = ""
        "value" = ""
    }
    ownership_verification_http = {
        "http_body" = ""
        "http_url"  = ""
    }
    zone_id                     = "6a9006164edb31aca836cf6f4d0c6184"

    ssl {
        certificate_authority = "digicert"
        method                = "txt"
        status                = "active"
        type                  = "dv"
        validation_errors     = []
        validation_records    = []
        wildcard              = false

        settings {
            ciphers         = []
            min_tls_version = "1.2"
        }
    }
}

Steps to reproduce

  1. Create a custom hostname
  2. Add the txt records to your DNS provider
  3. Refresh your terraform state

Additional factoids

We want to use the output of these records to make the txt records in AWS Route53. It works initially but since after the information is verified it is removed from the terraform state future runs attempt to change the AWS Route53 records since the values no longer exist in state.

References

No response

jacobbednarz commented 2 years ago

Thank you for reporting this issue! For maintainers to dig into issues it is required that all issues include the entirety of TF_LOG=DEBUG output to be provided. The only parts that should be redacted are your user credentials in the X-Auth-Key, X-Auth-Email and Authorization HTTP headers. Details such as zone or account identifiers are not considered sensitive but can be redacted if you are very cautious. This log file provides additional context from Terraform, the provider and the Cloudflare API that helps in debugging issues. Without it, maintainers are very limited in what they can do and may hamper diagnosis efforts.

This issue has been marked with triage/needs-information and is unlikely to receive maintainer attention until the log file is provided making this a complete bug report.

bdandoy commented 2 years ago

@jacobbednarz I have updated this issue with a full debug output

ghost commented 1 year ago

Hi all, I am experiencing a similar issue.

Terraform v1.3.6
on windows_amd64
+ provider registry.terraform.io/cloudflare/cloudflare v3.32.0

I am creating a cloudflare_custom_hostname resource, and using the ssl block values to populate a cloudflare_record resource:

resource "cloudflare_custom_hostname" "domain" {
  zone_id  = var.zone_id
  hostname = var.hostname
  custom_origin_server = var.origin_server
  wait_for_ssl_pending_validation = true

  ssl {
    method = "txt"
    wildcard = true
  }
}

resource "cloudflare_record" "domain" {
  zone_id =  var.zone_id
  name    = cloudflare_custom_hostname.domain.ssl[0].validation_records[0].txt_name
  value   = cloudflare_custom_hostname.domain.ssl[0].validation_records[0].txt_value
  type    = "TXT"
}

This works as expected when creating new resources. However, subsequent runs of terraform plan/terraform apply receive an error:

│ Error: Invalid index
│
│   on domains.tf line 16, in resource "cloudflare_record" "domain":
│   16:   name    = cloudflare_custom_hostname.domain.ssl[0].validation_records[0].txt_name
│     ├────────────────
│     │ cloudflare_custom_hostname.domain.ssl[0].validation_records is empty list of object
│
│ The given key does not identify an element in this collection value: the collection has no elements.
╵
╷
│ Error: Invalid index
│
│   on domains.tf line 17, in resource "cloudflare_record" "domain":
│   17:   value   = cloudflare_custom_hostname.domain.ssl[0].validation_records[0].txt_value
│     ├────────────────
│     │ cloudflare_custom_hostname.domain.ssl[0].validation_records is empty list of object
│
│ The given key does not identify an element in this collection value: the collection has no elements.

However, when running terraform state show 'cloudflare_custom_hostname.domain' it gives the following, which does have the ssl block populated as expected:

# cloudflare_custom_hostname.domain:
resource "cloudflare_custom_hostname" "domain" {
    custom_origin_server            = "redacted"
    hostname                        = "redacted"
    id                              = "redacted"
    ownership_verification          = {
        "name"  = "_cf-custom-hostname.redacted"
        "type"  = "txt"
        "value" = "redacted"
    }
    ownership_verification_http     = {
        "http_body" = "redacted"
        "http_url"  = "http://redacted/.well-known/cf-custom-hostname-challenge/redacted"
    }
    wait_for_ssl_pending_validation = true
    zone_id                         = "redacted"

    ssl {
        certificate_authority = "digicert"
        method                = "txt"
        status                = "pending_validation"
        type                  = "dv"
        validation_errors     = []
        validation_records    = [
            {
                cname_name   = ""
                cname_target = ""
                emails       = []
                http_body    = ""
                http_url     = ""
                txt_name     = "redacted"
                txt_value    = "redacted"
            },
        ]
        wildcard              = true

        settings {
            ciphers = []
        }
    }
}

Should I raise this as a new issue, or is this related to the above?

ghost commented 1 year ago

As a workaround to https://github.com/cloudflare/terraform-provider-cloudflare/issues/1466#issuecomment-1380593608 I used a try around the value throwing the error, and used ignore_changes for subsequent runs of terraform plan/terraform apply.

resource "cloudflare_record" "domain" {
  zone_id = var.zone_id
  name    = try(cloudflare_custom_hostname.domain.ssl[0].validation_records[0].txt_name, "")
  value   = try(cloudflare_custom_hostname.domain.ssl[0].validation_records[0].txt_value, "")
  type    = "TXT"

  lifecycle {
    ignore_changes = [
      name, value
    ]
  }
}