Schema validation fails when deleting schema

[pwmcintyre-shell]

Hi team

I have a schema compatibility issue, where I need to publish an incompatible version.

To resolve this, I figure I would simply re-create the schema and the whole topic ...

But instead, on deleting a schema, it seems to validate it first ... which fails for the above reason!

Seeking advice, and wondering why is it validating on delete?

Error

$ terraform destroy -auto-approve -input=false -var-file="variables/topics.tfvars" -target='module.topic["example"]'
module.topic["example"].confluent_kafka_topic.topic: Refreshing state... [id=lkc-6kwymq/example]
module.topic["example"].confluent_subject_config.subject: Refreshing state... [id=lsrc-pg9xny/example-value]
module.topic["example"].confluent_schema.schema: Refreshing state... [id=lsrc-pg9xny/example-value/latest]
╷
│ Warning: Resource targeting is in effect
│ 
│ You are creating a plan with the -target option, which means that the
│ result of this plan may not represent all of the changes requested by the
│ current configuration.
│ 
│ The -target option is not for routine use, and is provided only for
│ exceptional situations such as recovering from errors or mistakes, or when
│ Terraform specifically suggests to use it as part of an error message.
╵
╷
│ Error: error validating Schema: error validating a schema: [{errorType:"TYPE_CHANGED", description:"A type at path '#/properties/_version' is different between the old schema and the new schema'} {oldSchemaVersion: 5} {oldSchema: '{ ... }'} {validateFields: 'true', compatibility: 'FORWARD_TRANSITIVE'}]
│ 
│   with module.topic["example"].confluent_schema.schema,
│   on modules/kafka-topic/main.tf line 39, in resource "confluent_schema" "schema":
│   39: resource "confluent_schema" "schema" {
│ 
╵

Terraform

terraform {
  required_providers {
    confluent = {
      source  = "confluentinc/confluent"
      version = "~> 1.55.0"
    }
  }
}

variable "name" {
  type = string
}

variable "schema" {
  type = string
}

variable "compatibility_level" {
  type = string
}

resource "confluent_kafka_topic" "topic" {
  topic_name = var.name
  config = {
    "cleanup.policy" = "compact"
    "confluent.value.schema.validation" = "true"
    "max.message.bytes" = 10485760
  }
}

resource "confluent_subject_config" "subject" {
  subject_name        = "${ var.name }-value"
  compatibility_level = var.compatibility_level
}

resource "confluent_schema" "schema" {
  subject_name  = confluent_subject_config.subject.subject_name
  format        = "JSON"
  schema        = var.schema != null ? var.schema : "{}"
}

[pwmcintyre-shell]

additionally, i have now deleted the subject and all its schema versions via the API ... and when asking Terraform to "destroy" it ... it complains that it is not there (HTTP 404)

you would think that if you ask Terraform to delete something that doesn't exists ... that would be a non-error!

│ Error: error reading Schema: error reading Schema "lsrc-pg9xny/example-value/latest": error loading the latest Schema: error loading the latest Schema: 404 Not Found: Subject 'example-value' not found.

[pwmcintyre-shell]

For posterity, here's how I got around the issue ...

1. manually delete the subject and all versions
2. remove the subject and schema from state with:

   terraform state rm 'confluent_subject_config.subject'
   terraform state rm 'confluent_schema.schema'

[linouk23]

Thanks for creating this issue!

@pwmcintyre-shell could you try new attribute skip_validation_during_plan (you'll need to update your TF Provider to 1.79.0) and set it to true:

resource "confluent_schema" "schema" {
  subject_name  = confluent_subject_config.subject.subject_name
  format        = "JSON"
  schema        = var.schema != null ? var.schema : "{}"
  skip_validation_during_plan = true
}

and see whether it helps? Thank you!