confluentinc / terraform-provider-confluent

Terraform Provider for Confluent
Apache License 2.0
31 stars 64 forks source link

Unexcepted behavior while removing schema references and update schema within confluent_schema resource #443

Open S1M0NM opened 2 months ago

S1M0NM commented 2 months ago

Terraform Provider Version: 2.1.0 Terraform Version: 1.8.5 & 1.9.5

We are currently removing all schema references from our schemas since they cause a lot of problems with clients apart from java clients. In order to do this we are going to remove all references and only use full-schemas with all references resolved before registering them. Now i wanted to do this using terraform and noticed a unexpected bahvior within the confluent_schema resource.

Initial Terraform Configuration:

resource "confluent_schema" "example" {
  subject_name = example.schema.subject
  format       = "AVRO"
  schema       = file("file-path")

  credentials {
    key    = var.schema_registry_api_key.id
    secret = var.schema_registry_api_key.secret
  }
  rest_endpoint = var.schema_registry.rest_endpoint

  schema_registry_cluster {
    id = var.schema_registry.id
  }

  schema_reference {
    name = "common.reference.A"
    subject_name = "common.reference.A"
    version = 3
  }

  schema_reference {
    name = "example.reference.A"
    subject_name = "example.reference.A"
    version = 2
  }

I actually wanted to remove the references from the resource and point to the new version of the schema, but if I remove the references, the Terraform provider ignores the changes to the schema and produces the following plan output. Due to the removed references and the missing schema changes we are not able to apply the plan since the types are unknown to the schema registry.

resource "confluent_schema" "example" {
  subject_name = example.schema.subject
  format       = "AVRO"
  schema       = file("other-file-path") # updated file path

  credentials {
    key    = var.schema_registry_api_key.id
    secret = var.schema_registry_api_key.secret
  }
  rest_endpoint = var.schema_registry.rest_endpoint

  schema_registry_cluster {
    id = var.schema_registry.id
  }

  # removed references
}
  # module.example.confluent_schema.example will be updated in-place
  ~ resource "confluent_schema" "schema" {
        id                          = "lsrc-<id>/example.schema.subject/latest"

      - schema_reference {
          - name         = "common.reference.A" -> null
          - subject_name = "common.reference.A" -> null
          - version      = 3 -> null
        }
      - schema_reference {
          - name         = "example.reference.A" -> null
          - subject_name = "example.reference.A" -> null
          - version      = 2 -> null
        }
    }

However, if I leave the references untouched and only change the schema, the provider correctly notices the schema changes but due to the still existing references, we are unable to update the schema to due a Can't redefine error during apply

resource "confluent_schema" "example" {
  subject_name = example.schema.subject
  format       = "AVRO"
  schema       = file("other-file-path") # only updated file path

  credentials {
    key    = var.schema_registry_api_key.id
    secret = var.schema_registry_api_key.secret
  }
  rest_endpoint = var.schema_registry.rest_endpoint

  schema_registry_cluster {
    id = var.schema_registry.id
  }

  schema_reference {
    name = "common.reference.A"
    subject_name = "common.reference.A"
    version = 3
  }

  schema_reference {
    name = "example.reference.A"
    subject_name = "example.reference.A"
    version = 2
  }
# module.example.confluent_schema.example will be updated in-place
  ~ resource "confluent_schema" "schema" {
        id                          = "lsrc-<id>/example.schema.subject/latest"
      ~ schema                      = jsonencode(<updated schema content>)
    }

Expected plan result if references are removed and schema file path is updated:

resource "confluent_schema" "example" {
  subject_name = example.schema.subject
  format       = "AVRO"
  schema       = file("other-file-path") # updated file path

  credentials {
    key    = var.schema_registry_api_key.id
    secret = var.schema_registry_api_key.secret
  }
  rest_endpoint = var.schema_registry.rest_endpoint

  schema_registry_cluster {
    id = var.schema_registry.id
  }

  # removed schema references
# module.example.confluent_schema.example will be updated in-place
  ~ resource "confluent_schema" "schema" {
        id                          = "lsrc-<id>/example.schema.subject/latest"
      ~ schema                      = jsonencode(<updated schema content>)

        - schema_reference {
            - name         = "common.reference.A" -> null
            - subject_name = "common.reference.A" -> null
            - version      = 3 -> null
          }
        - schema_reference {
            - name         = "example.reference.A" -> null
            - subject_name = "example.reference.A" -> null
            - version      = 2 -> null
          }
    }