SovereignCloudStack / k8s-observability

Deployment manifests and knowledge base for the KaaS observability solution
https://scs.community/
Apache License 2.0
1 stars 0 forks source link

Investigate why helm reuse-values flag can not be used with custom values in helm updgrade process #54

Closed matofeder closed 5 months ago

matofeder commented 5 months ago

As a CSP operator I want to upgrade the monitoring helm release and use --reuse-values flag. It means I want to execute e.g.:

helm upgrade dnation-kubernetes-monitoring-stack dnationcloud/dnation-kubernetes-monitoring-stack --reuse-values -f custom-values.yaml

Currently, grafana won't start properly due to the following error: error="Datasource provisioning error: yaml: line 3: could not find expected ':'".

This error is probably caused by cluster-logs: null override. In the non --reuse-values scenarios i.e. in the Helm install/upgrade process everything works fine with null override, but in the case of upgrade with --reuse-values flag, the following is rendered by helm:

  datasource.yaml: |-
    apiVersion: 1
    datasources:
    null
    - access: proxy
      isDefault: true
      name: thanos
      type: prometheus
      url: http://thanos-query-frontend:9090

The null value should not be there, and caused Grafana error mentioned above

Probably related issue: https://github.com/helm/helm/issues/12637

matofeder commented 5 months ago

The new Helm flag --reset-then-reuse-values resolves the issue.

helm upgrade dnation-kubernetes-monitoring-stack dnationcloud/dnation-kubernetes-monitoring-stack --reset-then-reuse-values -f custom-values.yaml