How to make my helmfile DRY?

[max-rocket-internet]

I've read writing-helmfile.md but honestly still don't understand how it's supposed to work 😅

Here's my helmfile:

context: eks_cluster1

releases:

- chart: ../../charts/apps/app1
  labels:
    app: app1
  name: eu01-stg01-app1
  secrets:
  - ../../charts/apps/app1/values/eu01-stg01/secrets.yaml
  values:
  - ../../charts/apps/app1/values/eu01-stg01/values.yaml

- chart: ../../charts/apps/app2
  labels:
    app: app2
  name: eu01-stg01-app2
  secrets:
  - ../../charts/apps/app2/values/eu01-stg01/secrets.yaml
  values:
  - ../../charts/apps/app2/values/eu01-stg01/values.yaml

How imagine I have 30 apps, app1-30, how can I make this helmfile nice and DRY? Can I loop over a list somewhere?

[mumoshu]

Hey! Basically, it depends. That's because everyone has different goal even though the word is "DRY".

In this specific case, you have a single helmfile.yaml, a conventional naming rule for name, labels.app and a conventional directory structure on where to put local charts according to their names, and where to put values files and secrets files depending on the release names, right?

If so, I'd suggest extracting the common structure into a reusable go template:

{{ define "app" }}
- chart: ../../charts/apps/{{.App}}
  labels:
    app: {{.App}}
  name: {{.Zone}}-{{.App}}
  # snip
{{ end }}

releases:
{{ template "app" (dict "App" "app1" "Zone" "eu01-stg01")
{{ template "app" (dict "App" "app2" "Zone" "eu01-stg01")

[max-rocket-internet]

OK thank you @mumoshu, helpful as always 😃

You can close this issue if you wish.

[benmathews]

Hey! Basically, it depends. That's because everyone has different goal even though the word is "DRY".

In this specific case, you have a single helmfile.yaml, a conventional naming rule for name, labels.app and a conventional directory structure on where to put local charts according to their names, and where to put values files and secrets files depending on the release names, right?

If so, I'd suggest extracting the common structure into a reusable go template:

{{ define "app" }}
- chart: ../../charts/apps/{{.App}}
  labels:
    app: {{.App}}
  name: {{.Zone}}-{{.App}}
  # snip
{{ end }}

releases:
{{ template "app" (dict "App" "app1" "Zone" "eu01-stg01")
{{ template "app" (dict "App" "app2" "Zone" "eu01-stg01")

This example should be in the readme.

[max-rocket-internet]

@mumoshu FYI you are missing }} at the end of your last 2 lines.

Can we loop over a list of apps instead of having a line for app1 and app2? What's the syntax for that?

[mumoshu]

@max-rocket-internet Thanks for the correction!

Re: looping, it should look like:

releases:
{{ range $_, $app := (list "app1" "app2") }}
{{ template "app" (dict "App" $app "Zone" "eu01-stg01") }}
{{ end }}

[max-rocket-internet]

@mumoshu is there a way I could define multiple template in a single file and then call these templates from multiple helmfiles? Otherwise I see I have to define the same template in every helmfil, even though they are all the same.

[mumoshu]

@max-rocket-internet It isn't possible at the moment, unfortunately. A workaround exists though - try {{ tpl (readFile "template.yaml.tpl"} (dict "Values" (dict "foo" "bar")) }}!

[max-rocket-internet]

OK here's what I've done after this help from @mumoshu...

For context, we have around 400 releases, spread across 8 clusters and around 20 helmfiles.

Our (simplified) directory structure now looks like this:

helmfiles
├── infra-cluster1.yaml
├── main.yaml
├── prd.yaml
├── stg.yaml
└── templates
    ├── app.yaml
    ├── infra-charts.yaml

main.yaml:

helmfiles:
  - prd.yaml
  - stg.yaml
  - infra-cluster1.yaml

prd.yaml:

{{ $apps := list "app1" "app2" "app3" }}

releases:

{{ range $_, $app := $apps }}
{{ tpl (readFile "templates/app.yaml") (dict "App" $app "Env" "prd01" ) }}
{{ tpl (readFile "templates/app.yaml") (dict "App" $app "Env" "prd02" ) }}
{{ tpl (readFile "templates/app.yaml") (dict "App" $app "Env" "prd03" ) }}
{{ end }}

stg.yaml:

{{ $apps := list "app3" "app4" }}

releases:

{{ range $_, $app := $apps }}
{{ tpl (readFile "templates/app.yaml") (dict "App" $app "Env" "stg01" ) }}
{{ tpl (readFile "templates/app.yaml") (dict "App" $app "Env" "qa02" ) }}
{{ end }}

app.yaml:

- name: {{.Env}}-{{.App}}
  chart: ../charts/apps/{{.App}}
  kubeContext: {{.kubeContext}}
  labels:
    app: {{.App}}
    env: {{.Env}}
  values:
    - ../charts/apps/{{.App}}/values/{{.Env}}/values.yaml
  secrets:
    - ../charts/apps/{{.App}}/values/{{.Env}}/secrets.yaml

infra-cluster1.yaml:

{{ $cluster_name := "cluster1" }}

releases:

{{ tpl (readFile "templates/infra-charts.yaml") (dict "Cluster" $cluster_name "cluster_autoscaler_version" "3.2.0" "datadog_version" "1.32.1" "external_dns_version" "1.7.5" "ingress_version" "1.17.1" "metrics_server_version" "2.8.4" ) }}

infra-charts.yaml:

- name: cluster-autoscaler
  chart: stable/cluster-autoscaler
  kubeContext: {{.Cluster}}
  namespace: kube-system
  version: {{.cluster_autoscaler_version}}
  labels:
    app: cluster-autoscaler
    cluster: {{.Cluster}}
  values:
    - ../cluster-config/helm-value-files/cluster-autoscaler/{{.Cluster}}/values.yaml

- name: datadog
  chart: stable/datadog
  kubeContext: {{.Cluster}}
  namespace: kube-system
  version: {{.datadog_version}}
  labels:
    app: datadog
    cluster: {{.Cluster}}
  values:
    - ../cluster-config/helm-value-files/datadog/{{.Cluster}}/values.yaml
  secrets:
    - ../cluster-config/helm-value-files/datadog/{{.Cluster}}/secrets.yaml

- name: external-dns
  chart: stable/external-dns
  kubeContext: {{.Cluster}}
  namespace: kube-system
  version: {{.external_dns_version}}
  labels:
    app: external-dns
    cluster: {{.Cluster}}
  values:
    - ../cluster-config/helm-value-files/external-dns/{{.Cluster}}/values.yaml

- name: ingress01
  chart: stable/nginx-ingress
  kubeContext: {{.Cluster}}
  version: {{.ingress_version}}
  labels:
    app: nginx-ingress-private
    cluster: {{.Cluster}}
  values:
    - ../cluster-config/helm-value-files/nginx-ingress/{{.Cluster}}/values-private.yaml
...

This enabled us to go from around 4000 lines of YAML in helmfiles to around 300. Hopefully this can help someone else 😃

[max-rocket-internet]

@mumoshu do you know of a way to test if a Helm chart values file exists, then include it if so? i.e a condition that evaluates the existence of a file. I tried Files.Glob in a condition but it didn't work as expected.

[mumoshu]

@max-rocket-internet I believe missingFileHandler would help that.

missingFileHandler: Warn # set to either "Error" or "Warn". "Error" instructs helmfile to fail when unable to find a values or secrets file. When "Warn", it prints the file and continues. https://github.com/roboll/helmfile#configuration

[dudicoco]

Hi, I have a similar use case which I'm trying to work out. I am loading all of the helmfiles, each containing releases as a glob:

helmfiles:
- path: ../*/*/release-*.yaml 

I would like to template each of the releases when loading them. I don't want to add templates within the individual helmfiles as I consider them as sort of "value files for helmfile", which should be templated and rendered behind the scenes.

Any suggestions?

Thanks

[sigurdblueface]

Hi, @mumoshu , could you please clarify if there is a possibility to reference a value from values file in this kind of templates?

{{ define "app" }}
  - name: {{ .Name }}
    namespace: {{ .Namespace }}
    chart: chartrepo/{{ .Name }}
    version: {{ .Version }}
    labels:
      app: {{ .Name }}
      level: apps
    values:
      - {{ .Name }}/values.gotmpl
{{ end }}

releases:
{{ template "app" (dict "Name" "app1" "Namespace" `{{  .Values.Namespace }}` "Version" "1.0.7" )}}
{{ template "app" (dict "Name" "app2" "Namespace" `{{ .Values.Namespace }}` "Version" "1.0.3" )}}
{{ template "app" (dict "Name" "app3" "Namespace" `{{ .Values.Namespace }}` "Version" "1.0.4" )}}

the above code gives me line 15: cannot unmarshal !!map into string error placing .Values.namespace reference into the template expectedly results in error

executing "app" at <.Values.Namespace>: map has no entry for key "Values"

what's the correct syntax here?

╰─ helmfile --version                            
helmfile version v0.114.0

[mumoshu]

@sigurdblueface Sorry I can't get what you're trying. What should {{ .Values.Namespace }} result in for each apps(app1 to app3) you have?

[mumoshu]

Anyway, using bare "`" within go template seems impossible by its nature.

[sigurdblueface]

@mumoshu my goal is to have ability to specify a release namespace via values file

Project structure is:

myapps-deploy:
  myapps:
    helfmfile.yaml
    default.yaml
  helmfile.yaml
  production.yaml

'parent' helmfile:

helmfiles:
  - path: myapps/helmfile.yaml
    values:
      - myapps/default.yaml
      - production.yaml

'child' helmfile could be seen in my previous comment

well, let's say the default.yaml file is:

App1:
  Namespace: testns1
App2:
  Namespace: testns2
...

so I'd like the code below

releases:
{{ template "app" (dict "Name" "app1" "Namespace" "{{`{{  .Values.App1.Namespace }}`}}" "Version" "1.0.7" )}}
{{ template "app" (dict "Name" "app2" "Namespace" "{{`{{ .Values.App2.Namespace }}`}}" "Version" "1.0.3" )}}

to result in:

  - name: app1
    namespace: testns1
...
  - name: app2
    namespace: testns2

[mumoshu]

@sigurdblueface To me, it seems like you'd want to write:

{{ template "app" (dict "Name" "app1" "Namespace" .Values.App1.Namespace "Version" "1.0.7" )}}
{{ template "app" (dict "Name" "app2" "Namespace" .Values.App2.Namespace "Version" "1.0.3" )}}

[4c74356b41]

hey, sorry to bother you, but how you would do something like the person above, but I need to use range because I have 10+ apps:

releases:
{{ range $_, $app := (list "app1" "app2") }}
{{ template "app" (dict "App" $app "Zone" $.Environment.Values.{{ .app }}.tag ) }}
{{ end }}

the above works if I hardcode app1 instead of {{ .app }} but that, obviously, breaks app2. my environment looks like this:

environments:
  amazon:
    values:
    - app1:
        tag: xxx
    - app2:
        tag: yyy

update for anyone wondering:

releases:
{{ range $_, $app := (list "app1" "app2") }}
{{ $tmp := index $.Environment.Values $app }}
{{ template "app" (dict "App" $app "Zone" $tmp.tag ) }}
{{ end }}