oracle-terraform-modules / terraform-oci-oke

The Terraform OKE Module Installer for Oracle Cloud Infrastructure provides a Terraform module that provisions the necessary resources for Oracle Container Engine.
https://oracle-terraform-modules.github.io/terraform-oci-oke/
Universal Permissive License v1.0
153 stars 206 forks source link

Default values for cluster autoscaler helm chart cannot be overridden correctly #922

Closed VladislavGeneDx closed 4 months ago

VladislavGeneDx commented 4 months ago

Community Note

Terraform Version and Provider Version

$ tf version
Terraform v1.7.5
on darwin_amd64

Module version: "5.1.2"

Affected Resource(s)

module.oke.module.extensions[0].data.helm_template.cluster_autoscaler[0]

Terraform Configuration Files

module "oke" {
  source  = "oracle-terraform-modules/oke/oci"
  version = "5.1.2"

  providers = { oci.home = oci }

  ...
  # Cluster autoscaler environment variables
  cluster_autoscaler_install           = true
  cluster_autoscaler_namespace         = "kube-system"
  cluster_autoscaler_helm_version      = "9.37.0" 
  cluster_autoscaler_helm_values       = {
    "cloudProvider" = "oci",
    "image.tag"     = "1.29.0-10",
  }
  ...
}

Expected Behavior

We expected that the default values in the generated helm chart would be overridden by provided variables.

Actual Behavior

The provided values were ignored.

Steps to Reproduce

Provide to the oke module custom values like in the example below via cluster_autoscaler_helm_values variable.

Important Factoids

If you add variables to autoscaler helm chart it will override the default values only if the value of a variable is sorted lower in alphanumeric order then default one. It happens because the order of sets in helm_template doesn't affect the order of applied variables. It creates toset(), which sorts all values in alphanumeric order and uses the latest one.

terraform console
> toset([{ "name" = "cloudProvider", "type" = "", "value" = "oci" }, { "name" = "cloudProvider", "type" = "", "value" = "oci-oke" }])
toset([
  {
    "name" = "cloudProvider"
    "type" = ""
    "value" = "oci"
  },
  {
    "name" = "cloudProvider"
    "type" = ""
    "value" = "oci-oke"
  },
])
> toset([{ "name" = "cloudProvider", "type" = "", "value" = "oci-oke" }, { "name" = "cloudProvider", "type" = "", "value" = "oci" }])
toset([
  {
    "name" = "cloudProvider"
    "type" = ""
    "value" = "oci"
  },
  {
    "name" = "cloudProvider"
    "type" = ""
    "value" = "oci-oke"
  },
])
> toset([{ "name" = "cloudProvider", "type" = "", "value" = "a" }, { "name" = "cloudProvider", "type" = "", "value" = "b" }])
toset([
  {
    "name" = "cloudProvider"
    "type" = ""
    "value" = "a"
  },
  {
    "name" = "cloudProvider"
    "type" = ""
    "value" = "b"
  },
])
> toset([{ "name" = "cloudProvider", "type" = "", "value" = "b" }, { "name" = "cloudProvider", "type" = "", "value" = "a" }])
toset([
  {
    "name" = "cloudProvider"
    "type" = ""
    "value" = "a"
  },
  {
    "name" = "cloudProvider"
    "type" = ""
    "value" = "b"
  },

It's possible to fix by merging values before set: replace these lines with something like:

  dynamic "set" {
    for_each = merge(local.cluster_autoscaler_defaults, var.cluster_autoscaler_helm_values)
    iterator = helm_value
    content {
      name  = helm_value.key
      value = helm_value.value
    }
  }
hyder commented 4 months ago

Resolved in #926