Changing OIDC auth config of OKE cluster results in "Invalid OidcAuthnConfig.CaCertificate" error

alfredkrohmer commented 4 weeks ago

I'm trying to change the OIDC auth config on an enhanced OKE cluster that already has it enabled, more specifically I'm trying to switch to a different OIDC issuer and a different client ID.

  # oci_containerengine_cluster.main will be updated in-place
  ~ resource "oci_containerengine_cluster" "main" {
        id                            = "ocid1.cluster.oc1.iad.aaaaaaaa...redacted..."
        name                          = "...redacted..."
        # (10 unchanged attributes hidden)

      ~ options {
            # (1 unchanged attribute hidden)

          ~ open_id_connect_token_authentication_config {
              ~ client_id                       = "...redacted..." -> "kubelogin"
              ~ issuer_url                      = "https://login.microsoftonline.com/...redacted...v2.0" -> "https://dex....redacted..."
                # (7 unchanged attributes hidden)
            }

            # (5 unchanged blocks hidden)
        }

        # (3 unchanged blocks hidden)
    }

This results in the following error:

2024-10-24 08:13:54,065 INFO     Applying plan
Acquiring state lock. This may take a few moments...
oci_containerengine_cluster.main: Modifying... [id=ocid1.cluster.oc1.iad.aaaaaaaa...redacted...]
╷
│ Error: 400-InvalidParameter, Invalid OidcAuthnConfig.CaCertificate: Certificate processing error. Possibly invalid certificate data.
│ Suggestion: Please update the parameter(s) in the Terraform config as per error message Invalid OidcAuthnConfig.CaCertificate: Certificate processing error. Possibly invalid certificate data.
│ Documentation: https://registry.terraform.io/providers/oracle/oci/latest/docs/resources/containerengine_cluster 
│ API Reference:  
│ Request Target: PUT https://containerengine.us-ashburn-1.oci.oraclecloud.com/20180222/clusters/ocid1.cluster.oc1.iad.aaaaaaaa...redacted...
│ Provider version: 6.14.0, released on 2024-10-20.  
│ Service: Containerengine Cluster 
│ Operation Name: UpdateCluster 
│ OPC request ID: ...redacted... 
│ 
│ 
│   with oci_containerengine_cluster.main,
│   on oke-cluster.tf line 1, in resource "oci_containerengine_cluster" "main":
│    1: resource "oci_containerengine_cluster" "main" {
│ 
╵

However, if I put the following into a file options.json:

{
  "options": {
    "openIdConnectDiscovery": {
      "isOpenIdConnectDiscoveryEnabled": true,
      "issuerUrl": "https://dex....redacted...",
      "clientId": "kubelogin",
      "usernameClaim": "email",
      "usernamePrefix": "oidc:",
      "groupsClaim": "groups",
      "groupsPrefix": "oidc:"
     }
  }
}

The following command succeeds and does the expected change:

$ oci ce cluster update --options file://./options.json --cluster-id ocid1.cluster.oc1.iad.aaaaaaaa...redacted...

Hence this looks like a bug in the Terraform provider.

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version and Provider Version

$ terraform -v
Terraform v1.9.5
on linux_amd64

Your version of Terraform is out of date! The latest version
is 1.9.8. You can update by downloading from https://www.terraform.io/downloads.html

$ ./.terraform/providers/registry.terraform.io/oracle/oci/6.14.0/linux_amd64/terraform-provider-oci_v6.14.0
2024/10/24 08:29:12 [INFO] terraform-provider-oci 6.14.0
2024/10/24 08:29:12 Executable runs in Terraform plugin mode by default. For additional usage options, please run with the '-help' flag.
This binary is a plugin. These are not meant to be executed directly.
Please execute the program that consumes these plugins, which will
load any plugins automatically

Affected Resource(s)

oci_containerengine_cluster

Terraform Configuration Files

This the corresponding block in the oci_containerengine_cluster resource definition:

    dynamic "open_id_connect_token_authentication_config" {
      for_each = var.oidcAuthEnabled ? [1] : []
      content {
        is_open_id_connect_auth_enabled = true
        issuer_url                      = var.oidcIssuerURL
        client_id                       = var.oidcClientID
        username_claim                  = "email"
        username_prefix                 = "oidc:"
        groups_claim                    = "groups"
        groups_prefix                   = "oidc:"
      }
    }

Debug Output

Panic Output

Expected Behavior

Terraform should have update the OIDC auth parameters.

Actual Behavior

An error occurred.

Steps to Reproduce

terraform apply

Important Factoids

References

tf-oci-pub commented 4 weeks ago

Thank you for reporting the issue. We have raised an internal ticket to track this. Our service engineers will get back to you.

alfredkrohmer commented 4 weeks ago

I just noticed that this error seems to pop up for any change to the cluster that Terraform wants to perform once OIDC authentication is enabled. I get the exact same error if Terraform is trying to perform the following change on a cluster where OIDC authentication is enabled:

  # oci_containerengine_cluster.main will be updated in-place
  ~ resource "oci_containerengine_cluster" "main" {
        id                            = "ocid1.cluster.oc1.eu-frankfurt-1.aaaaaaaao...redacted..."
        name                          = "...redacted..."
        # (10 unchanged attributes hidden)

      ~ options {
            # (1 unchanged attribute hidden)

          + open_id_connect_discovery {
              + is_open_id_connect_discovery_enabled = true
            }

            # (6 unchanged blocks hidden)
        }

        # (3 unchanged blocks hidden)
    }

Error:

2024-10-24 10:16:28,972 INFO     Applying plan
Acquiring state lock. This may take a few moments...
oci_containerengine_cluster.main: Modifying... [id=ocid1.cluster.oc1.eu-frankfurt-1.aaaaaaaa...redacted...]
╷
│ Error: 400-InvalidParameter, Invalid OidcAuthnConfig.CaCertificate: Certificate processing error. Possibly invalid certificate data.
│ Suggestion: Please update the parameter(s) in the Terraform config as per error message Invalid OidcAuthnConfig.CaCertificate: Certificate processing error. Possibly invalid certificate data.
│ Documentation: https://registry.terraform.io/providers/oracle/oci/latest/docs/resources/containerengine_cluster 
│ API Reference:  
│ Request Target: PUT https://containerengine.eu-frankfurt-1.oci.oraclecloud.com/20180222/clusters/ocid1.cluster.oc1.eu-frankfurt-1.aaaaaaaa...redacted... 
│ Provider version: 6.14.0, released on 2024-10-20.  
│ Service: Containerengine Cluster 
│ Operation Name: UpdateCluster 
│ OPC request ID: ...redacted... 
│ 
│ 
│   with oci_containerengine_cluster.main,
│   on oke-cluster.tf line 1, in resource "oci_containerengine_cluster" "main":
│    1: resource "oci_containerengine_cluster" "main" {
│ 
╵

alfredkrohmer commented 4 weeks ago

Note: options.open_id_connect_token_authentication_config.ca_certificate is not set in the Terraform config as we use IDPs with certificates signed by recognized CAs.

gregvers commented 1 day ago

This issue is fixed. Thanks for reporting it.

oracle / terraform-provider-oci