[Bug] Cannot apply Terraform when appending a new certificate to accepted_client_ca while the change has been made

What are you really trying to do?

I am testing rotating certificates by appending a new one to accepted_client_ca.

For context, I created a Terraform module to integrate with secretmanager where we store the certificates. We have a job that rotates the certificates by appending to the secretmanager and we expect to do a terraform apply after the secret in secretmanager is updated. We want to hide these details so we created Terraform module which basically wrap the temporalcloud_namespace with aws_secretsmanager_secret.

data "aws_secretsmanager_secret" "certs" {
    name="${var.secret_name_prefix}/${var.environment}/${var.namespace}/certs"
}

data "aws_secretsmanager_secret_version" "certs" {
    secret_id=data.aws_secretsmanager_secret.certs.id
}

resource "temporalcloud_namespace" "temporal_namespace" {
    name=var.namespace
    regions=[var.region]
    accepted_client_ca=base64encode(data.aws_secretsmanager_secret_version.certs.secret_string)
    retention_days=var.retention_days
}

This is how I define the namespace:

module "cloud-ops-temporal-namespace" {
    source="./modules/temporal-namespace"
    namespace="cloud-ops"
    environment="prod"
    region="aws-us-east-1"
    retention_days=7
}

Describe the bug

After I set the initial certificates, I successfully set up my namespace using Terraform.

However, after I add a new certificate and apply again, there was an error:


Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # module.cloud-ops-temporal-namespace.temporalcloud_namespace.temporal_namespace will be updated in-place
  ~ resource "temporalcloud_namespace" "temporal_namespace" {
      ~ accepted_client_ca = (sensitive value)
      ~ endpoints          = {
          ~ grpc_address = "cloud-ops.xxxx.tmprl.cloud:7233" -> (known after apply)
          ~ web_address  = "https://cloud-ops.xxxxxxx.web.tmprl.cloud" -> (known after apply)
        } -> (known after apply)
        id                 = "cloud-ops.xxxxxxx"
        name               = "cloud-ops"
        # (2 unchanged attributes hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

module.cloud-ops-temporal-namespace.temporalcloud_namespace.temporal_namespace: Modifying... [id=cloud-ops.xxxxxxx]
module.cloud-ops-temporal-namespace.temporalcloud_namespace.temporal_namespace: Still modifying... [id=cloud-ops.xxxxxxx, 10s elapsed]
╷
│ Error: Provider produced inconsistent result after apply
│
│ When applying changes to module.cloud-ops-temporal-namespace.temporalcloud_namespace.temporal_namespace, provider "provider[\"registry.terraform.io/temporalio/temporalcloud\"]" produced
│ an unexpected new value: .accepted_client_ca: inconsistent values for sensitive attribute.
│
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.
╵

When I tried to apply again, it's a different error:


Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # module.cloud-ops-temporal-namespace.temporalcloud_namespace.temporal_namespace will be updated in-place
  ~ resource "temporalcloud_namespace" "temporal_namespace" {
      ~ accepted_client_ca = (sensitive value)
      ~ endpoints          = {
          ~ grpc_address = "cloud-ops.xxxxxxx.tmprl.cloud:7233" -> (known after apply)
          ~ web_address  = "https://cloud-ops.xxxxxxx.web.tmprl.cloud" -> (known after apply)
        } -> (known after apply)
        id                 = "cloud-ops.xxxxxxx"
        name               = "cloud-ops"
        # (2 unchanged attributes hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

module.cloud-ops-temporal-namespace.temporalcloud_namespace.temporal_namespace: Modifying... [id=cloud-ops.xxxxxxx]
╷
│ Error: Failed to update namespace
│
│   with module.cloud-ops-temporal-namespace.temporalcloud_namespace.temporal_namespace,
│   on modules/temporal-namespace/main.tf line 17, in resource "temporalcloud_namespace" "temporal_namespace":
│   17: resource "temporalcloud_namespace" "temporal_namespace" {
│
│ rpc error: code = InvalidArgument desc = nothing to change
╵

I went to the console to check my CA certificate of my namespace. There are actually two.

But right now I cannot apply the Terraform and there is always a drift.


  # module.cloud-ops-temporal-namespace.temporalcloud_namespace.temporal_namespace will be updated in-place
  ~ resource "temporalcloud_namespace" "temporal_namespace" {
      ~ accepted_client_ca = (sensitive value)
      ~ endpoints          = {
          ~ grpc_address = "cloud-ops.xxxxxxx.tmprl.cloud:7233" -> (known after apply)
          ~ web_address  = "https://cloud-ops.xxxxxxx.web.tmprl.cloud" -> (known after apply)
        } -> (known after apply)
        id                 = "cloud-ops.xxxxxxx"
        name               = "cloud-ops"
        # (2 unchanged attributes hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

Minimal Reproduction

Environment/Versions

OS and processor: MacBook Pro (M2 Max)
Temporal Version: N/A
Are you using Docker or Kubernetes or building Temporal from source? N/A

Terraform information:

Terraform v1.5.5
on darwin_arm64
+ provider registry.terraform.io/hashicorp/aws v5.50.0
+ provider registry.terraform.io/temporalio/temporalcloud v0.0.8

temporalio / terraform-provider-temporalcloud