Helmfile not receiving updates

[Meemaw]

Which Renovate are you using?

WhiteSource Renovate App

Which platform are you using?

GitHub.com

What would you like to do?

I would like helmfiles in my repository to receive automatic updated, e.g: https://github.com/insights-io/Insight/blob/master/infrastructure/k8/tracing/helmfile.yaml

There are multiple helmfiles in the repo but none of them receives any updates.

[rarkins]

If you view the log at app.renovatebot.com, do you see anything relevant? Extracts will be cached if the master commit isn't changed, so pick the log that happened after the last repo change to see maximum details.

[Meemaw]

Nothing relevant -- search over file does not contain any "helmfile" occurrence, except for the Github Action which is totally unrelated:

          {
            "depName": "mamezou-tech/setup-helmfile",
            "currentValue": "v0.6.0",
            "commitMessageTopic": "{{{depName}}} action",
            "datasource": "github-tags",
            "versioning": "docker",
            "depType": "action",
            "pinDigests": false,
            "depIndex": 1,
            "warnings": [],
            "fixedVersion": "v0.6.0",
            "sourceUrl": "https://github.com/mamezou-tech/setup-helmfile",
            "updates": []
          },

[rarkins]

In that case it sounds like you're viewing a cached extract.

[rarkins]

If you locate the log from when the repo was last parsed - or wait until after the next time you make a commit and Renovate runs, you should see messages similar to the below (which I got from running locally against a fork):

DEBUG: Matched 4 file(s) for manager helmfile: infrastructure/k8/development/helmfile.yaml, infrastructure/k8/monitoring/helmfile.yaml, infrastructure/k8/recording/helmfile.yaml, infrastructure/k8/tracing/helmfile.yaml (repository=renovate-tests/Insight)
...
DEBUG: Failed to parse helmfile helmfile.yaml (repository=renovate-tests/Insight)
       "err": {
         "name": "YAMLException",
         "reason": "bad indentation of a mapping entry",
         "mark": {
           "name": null,
           "buffer": "---\nrepositories:\n  - name: stable\n    url: https://charts.helm.sh/stable\n  - name: bitnami\n    url: https://charts.bitnami.com/bitnami\n  - name: elastic\n    url: https://helm.elastic.co\n  - name: kubernetes-dashboard\n    url: https://kubernetes.github.io/dashboard/\n  - name: incubator\n    url: https://charts.helm.sh/incubator\n  - name: runatlantis\n    url: https://runatlantis.github.io/helm-charts\n  - name: ingress-nginx\n    url: https://kubernetes.github.io/ingress-nginx\n\nhelmfiles:\n {{ if eq .Environment.Name \"production\" }}- path: \"./releases/cert-manager.yaml\"{{ end }}\n\nreleases:\n  - name: ingress-nginx\n    chart: ingress-nginx/ingress-nginx\n    version: 3.11.0\n    namespace: {{ .Values.infraNamespace }}\n    values:\n     - ingress-nginx.yaml\n\n  - name: metabase\n    chart: stable/metabase\n    version: 0.13.0\n    namespace: {{ .Values.infraNamespace }}\n    values:\n      - metabase.yaml\n      - ingress:\n          enabled: true\n          tls:\n            - hosts:\n              - \"metabase.{{ .Values.infraDomain }}\"\n              - \"www.metabase.{{ .Values.infraDomain }}\"\n              secretName: metabase-tls\n          hosts:\n            - \"metabase.{{ .Values.infraDomain }}\"\n            - \"www.metabase.{{ .Values.infraDomain }}\"\n          annotations:\n{{ toYaml .Values.ingressAnnotations | indent 12 }}\n\n  - name: fluentbit\n    chart: stable/fluent-bit\n    version: 2.10.1\n    namespace: {{ .Values.infraNamespace }}\n    values:\n      - fluentbit.yaml\n      - backend:\n          es:\n            host: \"elasticsearch-master.{{ .Values.appNamespace }}\"\n      - input:\n          tail:\n            parser: {{ .Values.fluentbitParser }}\n\n  - name: kibana\n    chart: elastic/kibana\n    version: 7.9.2\n    namespace: {{ .Values.infraNamespace }}\n    values:\n      - elasticsearchHosts: \"http://elasticsearch-master.{{ .Values.appNamespace }}:9200\"\n      - ingress:\n          enabled: true\n          tls:\n            - hosts:\n              - \"kibana.{{ .Values.infraDomain }}\"\n              secretName: kibana-tls\n          hosts:\n            - \"kibana.{{ .Values.infraDomain }}\"\n          annotations:\n{{ toYaml .Values.ingressAnnotations | indent 12 }}\n\n  - name: dashboard\n    chart: kubernetes-dashboard/kubernetes-dashboard\n    version: 2.8.3\n    namespace: {{ .Values.infraNamespace }}\n    values:\n      - dashboard.yaml\n      - ingress:\n          tls:\n            - hosts:\n              - \"dashboard.{{ .Values.infraDomain }}\"\n              - \"www.dashboard.{{ .Values.infraDomain }}\"\n              secretName: dashboard-tls\n          hosts:\n            - \"dashboard.{{ .Values.infraDomain }}\"\n            - \"www.dashboard.{{ .Values.infraDomain }}\"\n          annotations:\n{{ toYaml .Values.ingressAnnotations | indent 12 }}\n\n{{ if eq .Environment.Name \"production\" }}\n  - name: oauth2-proxy\n    chart: stable/oauth2-proxy\n    version: 3.2.3\n    namespace: {{ .Values.infraNamespace }}\n    values:\n      - oauth-proxy.yaml\n      - config:\n          existingSecret: {{ .Values.oauthExistingSecret }}\n  - name: atlantis\n    chart: runatlantis/atlantis\n    version: 3.12.4\n    namespace: {{ .Values.infraNamespace }}\n    values:\n      - atlantis.yaml\n      - ingress:\n          tls:\n            - hosts:\n              - \"atlantis.{{ .Values.infraDomain }}\"\n              secretName: atlantis-tls\n          host: \"atlantis.{{ .Values.infraDomain }}\"\n          annotations:\n{{ toYaml .Values.ingressAnnotations | indent 12 }}\n\n      - extraManifests:\n          - apiVersion: networking.k8s.io/v1beta1\n            kind: Ingress\n            spec:\n              rules:\n              - host: \"atlantis.{{ .Values.infraDomain }}\"\n                http:\n                  paths:\n                  - backend:\n                      serviceName: atlantis\n                      servicePort: 80\n                    path: /events\n              tls:\n              - hosts:\n                - \"atlantis.{{ .Values.infraDomain }}\"\n                secretName: atlantis-tls\n            metadata:\n              name: atlantis-public\n              annotations:\n{{ toYaml .Values.ingressAnnotationsHttps | indent 16 }}\n\n{{ end }}\n\nenvironments:\n  default:\n    values:\n      - ./default.yaml\n    missingFileHandler: Error\n  production:\n    values:\n      - ./production.yaml\n    missingFileHandler: Error\n\u0000",
           "position": 533,
           "line": 18,
           "column": 43
         },
         "message": "bad indentation of a mapping entry at line 19, column 44:\n     ... Environment.Name \"production\" }}- path: \"./releases/cert-manager ... \n                                         ^",
         "stack": "YAMLException: bad indentation of a mapping entry at line 19, column 44:\n     ... Environment.Name \"production\" }}- path: \"./releases/cert-manager ... \n                                         ^\n    at generateError (/Users/rhys/src/renovate/node_modules/js-yaml/lib/js-yaml/loader.js:167:10)\n    at throwError (/Users/rhys/src/renovate/node_modules/js-yaml/lib/js-yaml/loader.js:173:9)\n    at readBlockMapping (/Users/rhys/src/renovate/node_modules/js-yaml/lib/js-yaml/loader.js:1107:7)\n    at composeNode (/Users/rhys/src/renovate/node_modules/js-yaml/lib/js-yaml/loader.js:1359:12)\n    at readDocument (/Users/rhys/src/renovate/node_modules/js-yaml/lib/js-yaml/loader.js:1525:3)\n    at loadDocuments (/Users/rhys/src/renovate/node_modules/js-yaml/lib/js-yaml/loader.js:1588:5)\n    at load (/Users/rhys/src/renovate/node_modules/js-yaml/lib/js-yaml/loader.js:1614:19)\n    at Object.safeLoad (/Users/rhys/src/renovate/node_modules/js-yaml/lib/js-yaml/loader.js:1637:10)\n    at Object.extractPackageFile (/Users/rhys/src/renovate/lib/manager/helmfile/extract.ts:21:27)\n    at extractPackageFile (/Users/rhys/src/renovate/lib/manager/index.ts:94:5)\n    at getManagerPackageFiles (/Users/rhys/src/renovate/lib/workers/repository/extract/manager-files.ts:51:57)\n    at async /Users/rhys/src/renovate/lib/workers/repository/extract/index.ts:52:26\n    at async Promise.all (index 7)\n    at async extractAllDependencies (/Users/rhys/src/renovate/lib/workers/repository/extract/index.ts:50:26)\n    at async extract (/Users/rhys/src/renovate/lib/workers/repository/process/extract-update.ts:81:20)\n    at async extractDependencies (/Users/rhys/src/renovate/lib/workers/repository/process/index.ts:88:26)\n    at async Object.renovateRepository (/Users/rhys/src/renovate/lib/workers/repository/index.ts:39:52)\n    at async Object.start (/Users/rhys/src/renovate/lib/workers/global/index.ts:67:7)\n    at async /Users/rhys/src/renovate/lib/renovate.ts:10:22"
       },
       "fileName": "infrastructure/k8/development/helmfile.yaml"

[Meemaw]

Yes, I just found that myself. The UI is quite laggy on big logfiles 😞

Is this a bug in renovate YAML parsing? I can correctly apply those helmfiles without any issues.

[rarkins]

We rely on the js-yaml library to parse: https://github.com/renovatebot/renovate/blob/95681819c1212151dc8c9dd60c2604ddcd03eee0/lib/manager/helmfile/extract.ts#L2

If you think your YAML file is valid then best thing would be to raise an issue in their repo if there's not one already.

[viceice]

No, it's not a yaml issue, the file contains helm template token, so this is no longer a valid yaml file and parsing as yaml will always fail.

Current workaround is to use a regex manager.

[Meemaw]

Yes, so yaml does not support everthing helmfile does. It would work however work this way:

helmfile --file infrastructure/k8/monitoring/helmfile.yaml template | npx js-yaml

@viceice do you have any documentation on how to use it in this case?

[rarkins]

Is it proprietary helmfile templating or just leveraging Go templates? See https://github.com/renovatebot/renovate/issues/7333 for how we would probably handle this in future

[viceice]

Looks like the normal go template language to me

[Meemaw]

Yes its just go templating. Anyway, I think this is failing for many people who don't even realise Renovate was supposed to bump their charts (I checked and followed up with you guys by change) 😄

Go templating is becoming very popular and is probably used in almost every non trivial "hello world" infrastructure setup so it would be a great addition imo 👍

[rarkins]

Another alternative is to ditch the YAML parser and write a basic line-based parser that cares only about the image name and value.

[rarkins]

I've also added https://github.com/renovatebot/renovate/issues/7810 as a feature request to use a line-based parsing approach instead of YAML

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed soon if no further activity occurs. If this question is not done (either you plan to update it or are waiting on someone to respond) then please add a comment here to bump it and/or get the other person's attention. We aim to do our best to solve every problem. This bot is here to help us clean up issues which are no longer of use to the original poster, and not to close anything prematurely, so bump as you need!