local_retention_ms VS retention_ms

[wanisfahmyDE]

Hi Aiven team,

We are using the provider since some time now and while updating to the latest version, we get this error:

Error: 409: {"errors":[{"message":"Cannot set local retention when service does not have tiered storage enabled","status":409}],"message":"Cannot set local retention when service does not have tiered storage enabled"} -

with module.my_cluster.aiven_kafka_topic.kafka_topic["my_topic"],
  on .terraform/modules/my_cluster/topics.tf line 1, in resource "aiven_kafka_topic" "kafka_topic":

  1: resource "aiven_kafka_topic" "kafka_topic" {

our kafka topic configs looks like this:

resource "aiven_kafka_topic" "kafka_topic" {
  for_each = local.topics

  project                = data.aiven_project.aiven_project.project
  service_name           = aiven_kafka.aiven_kafka.service_name
  topic_name             = try(each.value.topic_name, each.key)
  partitions             = try(each.value.partitions, 5)
  replication            = try(each.value.replication, 3)
  termination_protection = try(each.value.termination_protection, false)

  config {
    retention_ms              = try(each.value.config.retention_ms, 21600000)
    cleanup_policy            = try(each.value.config.cleanup_policy, "delete")
    min_insync_replicas       = try(each.value.config.min_insync_replicas, try(each.value.replication, 3) - 1)
    segment_ms                = try(each.value.config.segment_ms, 604800000)
    segment_bytes             = try(each.value.config.segment_bytes, 209715200)
    delete_retention_ms       = try(each.value.config.delete_retention_ms, 86400000)
    max_compaction_lag_ms     = try(each.value.config.max_compaction_lag_ms, null)
    min_compaction_lag_ms     = try(each.value.config.min_compaction_lag_ms, null)
    min_cleanable_dirty_ratio = try(each.value.config.min_cleanable_dirty_ratio, 0.5)
    max_message_bytes         = try(each.value.config.max_message_bytes, 1048588)
    message_timestamp_type    = try(each.value.config.message_timestamp_type, "CreateTime")
    retention_bytes           = try(each.value.config.retention_bytes, -1)
  }

  timeouts {
    create = try(each.value.timeouts.create, "1m")
    read   = try(each.value.timeouts.read, "5m")
  }
}

it seems that aiven introduced a new field starting v4.13.0 called local_retention_ms and local_retention_bytes and I was wondering if these are replacements for the old retention_ms & retention_bytes?

Just to note, this cluster does not have tiered storage enabled.

Any ideas on how to fix this error? We still need to set these values correctly.

Thanks <3

[Serpentiel]

Hey, @wanisfahmyDE! 👋

Thank you for raising this question.

I will check things out and will let you know shortly 🙂

[Serpentiel]

Hey, @wanisfahmyDE! 👋

May I know what is the version that you're trying to upgrade to?

Thank you.

[wanisfahmyDE]

Hey @Serpentiel yes sure, from v4.9.3 to v4.14.0

[Serpentiel]

Hey, @wanisfahmyDE! 👋

Thank you for confirming it. Could you please try it again?

There was a bug in backend related to this, which recently was fixed.

[wanisfahmyDE]

Hey @Serpentiel, I can confirm that the issue is now fixed. Many thanks! Closing issue

[wanisfahmyDE]

Hey @Serpentiel, we are facing the same error again now... did anything change on aiven's side?

[christophstengel]

Some observation: It happens especially, when we try to remove termination protection for topics via IAC. Creating/Removing topics seems to work.

[Serpentiel]

Hey, @wanisfahmyDE! 👋

Have you ever changed anything about your Terraform configuration, e.g. upgrading to the latest version?

Does it fails in the exact same way like it did earlier?

Thank you.

[christophstengel]

Hi @Serpentiel , it seems failing in the same way. And yes, we are now using aiven terraform provider 4.15.0, so the bug seems to be there as well. (I'm in the same team as Wanis)

[Serpentiel]

Thank you, @christophstengel!

Could you please check if the issue persists by rolling back to v4.14.0?

We are currently trying to determine if it has to do with the Terraform Provider or no.

[christophstengel]

Same Error in 4.14

[Serpentiel]

Hey again, folks!

It appears that this error is related to certain services that hadn't received a maintenance upgrade after the backend fix was published.

We held an internal discussion about this issue and the conclusion was to suggest y'all contacting Aiven Support for further in-depth analysis of the problem and help with the affected services.

We have tried it out with the freshly-created service, and the problem doesn't seem to reproduce, although we were able to identify a potential minor improvement to the general flow for the Kafka Topic resource.

Hopefully this helps. Please let us know if you have anything else to add 🙏

[wanisfahmyDE]

hey @Serpentiel,

just fyi, our cluster is running the latest kafka version 3.6.1 and there is no pending maintenance (all green on the aiven console)

we will create a support ticket to take it from there nevertheless.

Cheers and thanks for helping out.