search

confluentinc / terraform-provider-confluent

Terraform Provider for Confluent
Apache License 2.0
118 stars 61 forks source link

Add support for new topic configuration fields - message.timestamp.after.max.ms and message.timestamp.before.max.ms #341

Closed Noel-Jones closed 6 months ago

Noel-Jones commented 6 months ago

Following on from #339 I just got around to updating my code to set the fields message.timestamp.after.max.ms and message.timestamp.before.max.ms to the default values. Now I get the message

Error: error updating Kafka Topic "lkc-xxxx/xxx.customer.updates": "message.timestamp.after.max.ms" topic setting is read-only and cannot be updated. Read https://registry.terraform.io/providers/confluentinc/confluent/latest/docs/resources/confluent_kafka_topic for more details.
│ 
│   with confluent_kafka_topic.active["Development-dev_cluster_0-xxx.customer.updates"],
│   on topic.tf line 24, in resource "confluent_kafka_topic" "active":
│   24: resource "confluent_kafka_topic" "active" {

The provider needs to be updated to support setting of these fields. I am raising this as a new issue because it is a specific issue with a more significant impact that prevents me from being able to apply. If I set the config value I get the error shown above, if I don't set the field I get the error below!

Error: error updating Kafka Topic "lkc-xxxxx/dlq-oracle-erp.fct.customer.updates": reset to topic setting's default value operation (in other words, removing topic settings from 'configs' block) is not supported at the moment. Instead, find its default value at https://registry.terraform.io/providers/confluentinc/confluent/latest/docs/resources/confluent_kafka_topic and set its current value to the default value.
shriman-desikan commented 6 months ago

We are also having this issue, it's preventing us from updating any existing topics using Terraform. This is definitely a bug versus an enhancement since it prevents you from updating any existing topics and the Error messages' fix also doesn't work.

linouk23 commented 6 months ago

@Noel-Jones, @shriman-desikan thanks for creating this PR!

Could you confirm you are interested in the following 2 topic settings:

image

shriman-desikan commented 6 months ago

Yes, those are the two topic settings that I guess should be editable going forward.

This all seems related to this KIP and Confluent's adoption of Kafka version 3.6.x, for more context.

Thank you!

linouk23 commented 6 months ago

Thanks!

We have internally merged a fix and are planning to release a new version of the TF Provider today or tomorrow, which will include this fix. Thank you for your patience!

linouk23 commented 6 months ago

@shriman-desikan @Noel-Jones

Update: we

Added a client side fix for "Add support for new topic configuration fields" issue (https://github.com/confluentinc/terraform-provider-confluent/issues/341).

in our latest 1.57.0 version of TF Provider.

Thanks for waiting and let us know if you run into any issues when using the latest 1.57.0 version of TF Provider.

Noel-Jones commented 5 months ago

Thank you, v1.57 does resolve this issue.