Unexpected CSR replacement on v4.0.1

[benhoskings]

Terraform CLI and Provider Versions

Terraform v1.2.7
on darwin_amd64
+ provider registry.terraform.io/datadog/datadog v3.8.1
+ provider registry.terraform.io/hashicorp/archive v2.2.0
+ provider registry.terraform.io/hashicorp/aws v4.7.0
+ provider registry.terraform.io/hashicorp/google v4.19.0
+ provider registry.terraform.io/hashicorp/google-beta v4.19.0
+ provider registry.terraform.io/hashicorp/http v3.0.1
+ provider registry.terraform.io/hashicorp/local v2.2.3
+ provider registry.terraform.io/hashicorp/null v3.1.1
+ provider registry.terraform.io/hashicorp/random v3.3.2
+ provider registry.terraform.io/hashicorp/tls v4.0.1

Terraform Configuration

resource "tls_private_key" "example" {
  algorithm = "ECDSA"
  ecdsa_curve = "P256"
}

resource "tls_cert_request" "example" {
  private_key_pem = tls_private_key.example.private_key_pem
  dns_names = []
  ip_addresses = []
  subject {
    common_name = "example"
  }
}

Expected Behavior

No diff

Actual Behavior

Plan recreates the tls_cert_request

Steps to Reproduce

terraform plan on v4

How much impact is this issue causing?

High

Logs

No response

Additional Information

On upgrading to v4 of the tls provider I found that our tls_cert_request resources, which are unchanged and have no diff on v3.4, are up for replacement due to a change in the handling of the subject block. It appears that the block itself is forcing replacement, rather than any attribute within it.

Using tls v3.4.0, everything is as expected:

No changes. Your infrastructure matches the configuration.

Using tls v4.0.1, the subject block causes the resource to churn:

  # tls_cert_request.example must be replaced
-/+ resource "tls_cert_request" "example" {
      ~ cert_request_pem = <<-EOT
            -----BEGIN CERTIFICATE REQUEST-----
            MIHHMG8CAQA...
            -----END CERTIFICATE REQUEST-----
        EOT -> (known after apply)
      ~ id               = "3101d515..." -> (known after apply)
      ~ key_algorithm    = "ECDSA" -> (known after apply)
      ~ private_key_pem  = (sensitive value)
        # (2 unchanged attributes hidden)

      ~ subject { # forces replacement
          - street_address = [] -> null # forces replacement
            # (1 unchanged attribute hidden)
        }
    }

Using tls v4.0.1 and adding street_address = [] to the resource definition, the subject block still forces replacement even though none of the attributes within it are changing:

  # tls_cert_request.example must be replaced
-/+ resource "tls_cert_request" "example" {
      ~ cert_request_pem = <<-EOT
            -----BEGIN CERTIFICATE REQUEST-----
            MIHHMG8CAQA...
            -----END CERTIFICATE REQUEST-----
        EOT -> (known after apply)
      ~ id               = "3101d515..." -> (known after apply)
      ~ key_algorithm    = "ECDSA" -> (known after apply)
      ~ private_key_pem  = (sensitive value)
        # (2 unchanged attributes hidden)

      ~ subject { # forces replacement
            # (2 unchanged attributes hidden)
        }
    }

Using tls v4.0.1 and ignoring changes on all of subject, it looks OK; considering private_key_pem is now stored as-is this is an expected change:

# tls_cert_request.example will be updated in-place
~ resource "tls_cert_request" "example" {
      id               = "3101d515..."
    ~ private_key_pem  = (sensitive value)
      # (4 unchanged attributes hidden)

      # (1 unchanged block hidden)
  }

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[dpkirchner]

A partial workaround for this is to pull down your state file (terraform state pull), find the tls_cert_request.example resource, and copy the values in the subject object over to your .tf files. For example:

"subject": [
  {
    "common_name": "clientname",
    "country": "",
    "locality": "",
    "organization": "my org name",
    "organizational_unit": "",
    "postal_code": "",
    "province": "",
    "serial_number": "",
    "street_address": []
  }
],

becomes

  subject {
    common_name         = "clientname"
    organization        = "my org name"
    country             = ""
    locality            = ""
    organizational_unit = ""
    postal_code         = ""
    province            = ""
    serial_number       = ""
    street_address      = []
  }

You may still see ~ private_key_pem = (sensitive value) but at least it changes from a replacement to an update. (It's still not something I'm comfortable applying but maybe this'll help others that are OK with the private key changing).

[dpkirchner]

I wonder if the private_key_pem update is a red herring. I'm seeing changes to cert_request_pem that are reported as changes from a checksum to a CSR:

      ~ ca_cert_pem           = (sensitive)
      ~ ca_private_key_pem    = (sensitive value)
      ~ cert_request_pem      = <<-EOT
          - 56683b36b91b66c5b78cfdcfac4817f8a304b04d
          + -----BEGIN CERTIFICATE REQUEST-----
          + MIICdjCCAV4CAQAwMTESMBAGA1...

I've verified that the checksum (566..) is indeed the shasum for the CSR, so won't actually change anything. I think it's the same thing for private_key_pem based on the output of terraform show -out=json.

FWIW, I've tested this with v4.0.2.

[dpkirchner]

I've committed a fix to a local copy of the provider. It seems to work but I don't know if it is doing it right.

The problem: The state file contains a SHA1 hash of some PEMs, so when the plan is compared against the state a difference is identified. I think this only occurs when some other value in the resource changes (e.g. the street_address or whatever).

The "fix": If the SHA1 of the PEM in the plan matches the value in state, ignore the change. Or, rather, overwrite the plan value with the state value.

https://github.com/dpkirchner/terraform-provider-tls/commit/6610b48339251661ab112aa23215d128d7736e0f

[jwitko]

We're having the same issue currently. Upgrading to hashicorp/tls without any values changing is wanting to replace our entire root trust. Using 4.0.3

[BornToBeRoot]

Any update on this?