fluxcd / helm-controller

The GitOps Toolkit Helm reconciler, for declarative Helming
https://fluxcd.io
Apache License 2.0
406 stars 160 forks source link

HelmRelease fails when using fromValues with kind ConfigMap #486

Closed quintonanderson-zetaris closed 2 years ago

quintonanderson-zetaris commented 2 years ago

I am currently trying to source values for a HelmRelease from an existing ConfigMap within the cluster. The HelmRelease is being used to install nginx ingress controller, against a AKS cluster with a static IP address. As a result of the static external IP address, the ingress controller needs to know the public IP address and its associated resource group.

When I statically assign the values, everything works fine:

apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
  name: ingress-nginx
  namespace: cluster-config
spec:
  targetNamespace: kube-system
  releaseName: ingress
  chart:
    spec:
      chart: ingress-nginx
      version: 4.0.6
      sourceRef:
        kind: HelmRepository
        name: nginx
  interval: 1h0m0s
  values:
    rbac.create: true
    rbac.createRole: true
    controller.service.loadBalancerIP: x.x.x.x
    controller.service.annotations."service.beta.kubernetes.io/azure-load-balancer-resource-group": RESOURCE_GROUP_NAME
    controller.config.proxy-body-size: 100m
    controller.config.server-tokens: false
    service.type: NodePort

The IP address and resource name are created by Terraform and stored into a ConfigMap using the following Terraform resource:

resource "kubernetes_config_map" "cluster-metadata" {
  metadata {
    name = "cluster-metadata"
    namespace = "cluster-config"
  }

  data = {
    public_ip = var.public_ip
    resource_group_name = var.resource_group_name
  }
  depends_on = [azurerm_kubernetes_cluster.datafabric]
}

Therefore at cluster creation time, a configMap is created to with the Terraform managed state, which Flux can then use to manage Kubernetes level resources. Here you can see the resulting ConfigMap:

apiVersion: v1
data:
  public_ip: x.x.x.x
  resource_group_name: RESOURCE_GROUP_NAME
kind: ConfigMap
metadata:
  creationTimestamp: "2022-05-17T02:24:32Z"
  name: cluster-metadata
  namespace: cluster-config
  resourceVersion: "41319"
  uid: ...

However when I try to use the valuesFrom, I get the following error: Helm upgrade failed: YAML parse error on ingress-nginx/templates/controller-service.yaml: error converting YAML to JSON: yaml: line 4: did not find expected key.... Here is the snippet of relevant config (replacing the static values above):

valuesFrom:
    - kind: ConfigMap
      name: cluster-metadata
      valuesKey: public_ip
      targetPath: controller.service.loadBalancerIP
    - kind: ConfigMap
      name: cluster-metadata
      valuesKey: resource_group_name
      targetPath: controller.service.annotations."service.beta.kubernetes.io/azure-load-balancer-resource-group"
quintonanderson-zetaris commented 2 years ago

I figured out that this issue relates to #460, therefore updating the annotations targetPath fixes the issue: targetPath: controller.service.annotations."service\.beta\.kubernetes\.io/azure-load-balancer-resource-group"