gitlab-ci-pipelines-exporter: Can not disable default readiness and liveness probes by replacing them, the new ones from helm values are always added along with them.

[caermeglaeddyv]

my values:

image:
  pullPolicy: Always

fullnameOverride: pipelines-exporter

config:
  log:
    format: json
  gitlab:
    url: https://my-gitlab.local
    token: some_token
    enable_health_check: false
  projects:
  - name: some_group/some_project

serviceMonitor:
  enabled: true

livenessProbe: {}

readinessProbe: {}

when I create helm templates using that values here's what I get:

---
---
# Source: gitlab-ci-pipelines-exporter/charts/redis/templates/serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
automountServiceAccountToken: true
metadata:
  name: pipelines-exporter-redis
  namespace: "default"
  labels:
    app.kubernetes.io/name: redis
    helm.sh/chart: redis-14.6.1
    app.kubernetes.io/instance: pipelines-exporter
    app.kubernetes.io/managed-by: Helm
---
# Source: gitlab-ci-pipelines-exporter/templates/secret.yaml
apiVersion: v1
kind: Secret
metadata:
  name: pipelines-exporter-config
  labels:
    app.kubernetes.io/name: gitlab-ci-pipelines-exporter
    app.kubernetes.io/version: v0.5.1
    helm.sh/chart: gitlab-ci-pipelines-exporter-0.2.6
    app.kubernetes.io/instance: pipelines-exporter
    app.kubernetes.io/managed-by: Helm
type: Opaque
data:
  GCPE_GITLAB_TOKEN: c29tZV90b2tlbg==
  GCPE_REDIS_URL: cmVkaXM6Ly9waXBlbGluZXMtZXhwb3J0ZXItcmVkaXMtbWFzdGVyLmRlZmF1bHQuc3ZjOjYzNzk=
---
# Source: gitlab-ci-pipelines-exporter/charts/redis/templates/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: pipelines-exporter-redis-configuration
  namespace: "default"
  labels:
    app.kubernetes.io/name: redis
    helm.sh/chart: redis-14.6.1
    app.kubernetes.io/instance: pipelines-exporter
    app.kubernetes.io/managed-by: Helm
data:
  redis.conf: |-
    # User-supplied common configuration:
    # Enable AOF https://redis.io/topics/persistence#append-only-file
    appendonly yes
    # Disable RDB persistence, AOF persistence already enabled.
    save ""
    # End of common configuration
  master.conf: |-
    dir /data
    # User-supplied master configuration:
    rename-command FLUSHDB ""
    rename-command FLUSHALL ""
    # End of master configuration
  replica.conf: |-
    dir /data
    slave-read-only yes
    # User-supplied replica configuration:
    rename-command FLUSHDB ""
    rename-command FLUSHALL ""
    # End of replica configuration
---
# Source: gitlab-ci-pipelines-exporter/charts/redis/templates/health-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: pipelines-exporter-redis-health
  namespace: "default"
  labels:
    app.kubernetes.io/name: redis
    helm.sh/chart: redis-14.6.1
    app.kubernetes.io/instance: pipelines-exporter
    app.kubernetes.io/managed-by: Helm
data:
  ping_readiness_local.sh: |-
    #!/bin/bash

    [[ -f $REDIS_PASSWORD_FILE ]] && export REDIS_PASSWORD="$(< "${REDIS_PASSWORD_FILE}")"
    export REDISCLI_AUTH="$REDIS_PASSWORD"
    response=$(
      timeout -s 3 $1 \
      redis-cli \
        -h localhost \
        -p $REDIS_PORT \
        ping
    )
    if [ "$response" != "PONG" ]; then
      echo "$response"
      exit 1
    fi
  ping_liveness_local.sh: |-
    #!/bin/bash

    [[ -f $REDIS_PASSWORD_FILE ]] && export REDIS_PASSWORD="$(< "${REDIS_PASSWORD_FILE}")"
    export REDISCLI_AUTH="$REDIS_PASSWORD"
    response=$(
      timeout -s 3 $1 \
      redis-cli \
        -h localhost \
        -p $REDIS_PORT \
        ping
    )
    if [ "$response" != "PONG" ] && [ "$response" != "LOADING Redis is loading the dataset in memory" ]; then
      echo "$response"
      exit 1
    fi
  ping_readiness_master.sh: |-
    #!/bin/bash

    [[ -f $REDIS_MASTER_PASSWORD_FILE ]] && export REDIS_MASTER_PASSWORD="$(< "${REDIS_MASTER_PASSWORD_FILE}")"
    export REDISCLI_AUTH="$REDIS_MASTER_PASSWORD"
    response=$(
      timeout -s 3 $1 \
      redis-cli \
        -h $REDIS_MASTER_HOST \
        -p $REDIS_MASTER_PORT_NUMBER \
        ping
    )
    if [ "$response" != "PONG" ]; then
      echo "$response"
      exit 1
    fi
  ping_liveness_master.sh: |-
    #!/bin/bash

    [[ -f $REDIS_MASTER_PASSWORD_FILE ]] && export REDIS_MASTER_PASSWORD="$(< "${REDIS_MASTER_PASSWORD_FILE}")"
    export REDISCLI_AUTH="$REDIS_MASTER_PASSWORD"
    response=$(
      timeout -s 3 $1 \
      redis-cli \
        -h $REDIS_MASTER_HOST \
        -p $REDIS_MASTER_PORT_NUMBER \
        ping
    )
    if [ "$response" != "PONG" ] && [ "$response" != "LOADING Redis is loading the dataset in memory" ]; then
      echo "$response"
      exit 1
    fi
  ping_readiness_local_and_master.sh: |-
    script_dir="$(dirname "$0")"
    exit_status=0
    "$script_dir/ping_readiness_local.sh" $1 || exit_status=$?
    "$script_dir/ping_readiness_master.sh" $1 || exit_status=$?
    exit $exit_status
  ping_liveness_local_and_master.sh: |-
    script_dir="$(dirname "$0")"
    exit_status=0
    "$script_dir/ping_liveness_local.sh" $1 || exit_status=$?
    "$script_dir/ping_liveness_master.sh" $1 || exit_status=$?
    exit $exit_status
---
# Source: gitlab-ci-pipelines-exporter/charts/redis/templates/scripts-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: pipelines-exporter-redis-scripts
  namespace: "default"
  labels:
    app.kubernetes.io/name: redis
    helm.sh/chart: redis-14.6.1
    app.kubernetes.io/instance: pipelines-exporter
    app.kubernetes.io/managed-by: Helm
data:
  start-master.sh: |
    #!/bin/bash

    [[ -f $REDIS_PASSWORD_FILE ]] && export REDIS_PASSWORD="$(< "${REDIS_PASSWORD_FILE}")"
    if [[ ! -f /opt/bitnami/redis/etc/master.conf ]];then
        cp /opt/bitnami/redis/mounted-etc/master.conf /opt/bitnami/redis/etc/master.conf
    fi
    if [[ ! -f /opt/bitnami/redis/etc/redis.conf ]];then
        cp /opt/bitnami/redis/mounted-etc/redis.conf /opt/bitnami/redis/etc/redis.conf
    fi
    ARGS=("--port" "${REDIS_PORT}")
    ARGS+=("--protected-mode" "no")
    ARGS+=("--include" "/opt/bitnami/redis/etc/redis.conf")
    ARGS+=("--include" "/opt/bitnami/redis/etc/master.conf")
    exec redis-server "${ARGS[@]}"
---
# Source: gitlab-ci-pipelines-exporter/templates/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: pipelines-exporter-config
  labels:
    app.kubernetes.io/name: gitlab-ci-pipelines-exporter
    app.kubernetes.io/version: v0.5.1
    helm.sh/chart: gitlab-ci-pipelines-exporter-0.2.6
    app.kubernetes.io/instance: pipelines-exporter
    app.kubernetes.io/managed-by: Helm
data:
  config.yml: |
    log:
      level: info
      format: json
    gitlab:
      url: https://my-gitlab.local
      enable_health_check: true
      enable_tls_verify: true
    projects: 
      - name: some_group/some_project
---
# Source: gitlab-ci-pipelines-exporter/charts/redis/templates/headless-svc.yaml
apiVersion: v1
kind: Service
metadata:
  name: pipelines-exporter-redis-headless
  namespace: "default"
  labels:
    app.kubernetes.io/name: redis
    helm.sh/chart: redis-14.6.1
    app.kubernetes.io/instance: pipelines-exporter
    app.kubernetes.io/managed-by: Helm
spec:
  type: ClusterIP
  clusterIP: None
  ports:
    - name: tcp-redis
      port: 6379
      targetPort: redis
  selector:
    app.kubernetes.io/name: redis
    app.kubernetes.io/instance: pipelines-exporter
---
# Source: gitlab-ci-pipelines-exporter/charts/redis/templates/master/service.yaml
apiVersion: v1
kind: Service
metadata:
  name: pipelines-exporter-redis-master
  namespace: "default"
  labels:
    app.kubernetes.io/name: redis
    helm.sh/chart: redis-14.6.1
    app.kubernetes.io/instance: pipelines-exporter
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: master
spec:
  type: ClusterIP

  ports:
    - name: tcp-redis
      port: 6379
      targetPort: redis
      nodePort: null
  selector:
    app.kubernetes.io/name: redis
    app.kubernetes.io/instance: pipelines-exporter
    app.kubernetes.io/component: master
---
# Source: gitlab-ci-pipelines-exporter/templates/service.yaml
apiVersion: v1
kind: Service
metadata:
  name: pipelines-exporter
  labels:
    app.kubernetes.io/name: gitlab-ci-pipelines-exporter
    app.kubernetes.io/version: v0.5.1
    helm.sh/chart: gitlab-ci-pipelines-exporter-0.2.6
    app.kubernetes.io/instance: pipelines-exporter
    app.kubernetes.io/managed-by: Helm
    helm.sh/from: deploy.pipelines-exporter
spec:
  type: ClusterIP
  ports:
    - name: http
      port: 8080
      targetPort: 8080
      protocol: TCP
  selector:
    app.kubernetes.io/name: gitlab-ci-pipelines-exporter
    app.kubernetes.io/instance: pipelines-exporter
    helm.sh/from: deploy.pipelines-exporter
---
# Source: gitlab-ci-pipelines-exporter/templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: pipelines-exporter
  labels:
    app.kubernetes.io/name: gitlab-ci-pipelines-exporter
    app.kubernetes.io/version: v0.5.1
    helm.sh/chart: gitlab-ci-pipelines-exporter-0.2.6
    app.kubernetes.io/instance: pipelines-exporter
    app.kubernetes.io/managed-by: Helm

spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: gitlab-ci-pipelines-exporter
      helm.sh/from: deploy.pipelines-exporter
  strategy: 
    type: RollingUpdate
  template:
    metadata:
      labels:
        app.kubernetes.io/name: gitlab-ci-pipelines-exporter
        app.kubernetes.io/version: v0.5.1
        helm.sh/chart: gitlab-ci-pipelines-exporter-0.2.6
        app.kubernetes.io/instance: pipelines-exporter
        app.kubernetes.io/managed-by: Helm
        helm.sh/from: deploy.pipelines-exporter

      annotations:
        checksum/configmap: 2bc7d76d6caf7b894f0549cad98181e3c70802facff0d038c3ca0da989485d62
        checksum/secret: 057acf445aed4637bb75079beed534c330fef9b5433732f7d1d46b1697885298

    spec:
      containers:
        - name: gitlab-ci-pipelines-exporter
          image: mvisonneau/gitlab-ci-pipelines-exporter:v0.5.1
          imagePullPolicy: Always
          command: 
                - gitlab-ci-pipelines-exporter
                - run
          args: 
                - --config
                - /etc/config.yml
          env: 
                - name: GCPE_INTERNAL_MONITORING_LISTENER_ADDRESS
                  value: tcp://127.0.0.1:8082
          envFrom:
            - secretRef:
                name: pipelines-exporter-config
          volumeMounts:
            - name: config
              mountPath: /etc/config.yml
              subPath: config.yml
          ports:
            - name: exporter
              containerPort: 8080
              protocol: TCP
          livenessProbe: 
                httpGet:
                  path: /health/live
                  port: 8080
          readinessProbe: 
                failureThreshold: 3
                httpGet:
                  path: /health/ready
                  port: 8080
                initialDelaySeconds: 5
                periodSeconds: 30
                timeoutSeconds: 5
      volumes:
        - name: config
          configMap:
            name: pipelines-exporter-config
---
# Source: gitlab-ci-pipelines-exporter/charts/redis/templates/master/statefulset.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: pipelines-exporter-redis-master
  namespace: "default"
  labels:
    app.kubernetes.io/name: redis
    helm.sh/chart: redis-14.6.1
    app.kubernetes.io/instance: pipelines-exporter
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: master
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: redis
      app.kubernetes.io/instance: pipelines-exporter
      app.kubernetes.io/component: master
  serviceName: pipelines-exporter-redis-headless
  updateStrategy:
    rollingUpdate: {}
    type: RollingUpdate
  template:
    metadata:
      labels:
        app.kubernetes.io/name: redis
        helm.sh/chart: redis-14.6.1
        app.kubernetes.io/instance: pipelines-exporter
        app.kubernetes.io/managed-by: Helm
        app.kubernetes.io/component: master
      annotations:
        checksum/configmap: 9f9697d404a7c252fc4023ce44a21bbcff2ffa395416f909e65c587751968315
        checksum/health: 51d2aa2f30326eea679900b04f30a770f8bae1fa215a87cfce71805b82fa7167
        checksum/scripts: a61acb812feaaab0fc75a3c17d01fdae1c049d20362630528344511de4575161
        checksum/secret: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
    spec:

      securityContext:
        fsGroup: 1001
      serviceAccountName: pipelines-exporter-redis
      affinity:
        podAffinity:

        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
            - podAffinityTerm:
                labelSelector:
                  matchLabels:
                    app.kubernetes.io/name: redis
                    app.kubernetes.io/instance: pipelines-exporter
                    app.kubernetes.io/component: master
                namespaces:
                  - "default"
                topologyKey: kubernetes.io/hostname
              weight: 1
        nodeAffinity:

      terminationGracePeriodSeconds: 30
      containers:
        - name: redis
          image: docker.io/bitnami/redis:6.2.4-debian-10-r11
          imagePullPolicy: "IfNotPresent"
          securityContext:
            runAsUser: 1001
          command:
            - /bin/bash
          args:
            - -c
            - /opt/bitnami/scripts/start-scripts/start-master.sh
          env:
            - name: BITNAMI_DEBUG
              value: "false"
            - name: REDIS_REPLICATION_MODE
              value: master
            - name: ALLOW_EMPTY_PASSWORD
              value: "yes"
            - name: REDIS_TLS_ENABLED
              value: "no"
            - name: REDIS_PORT
              value: "6379"
          ports:
            - name: redis
              containerPort: 6379
          livenessProbe:
            initialDelaySeconds: 5
            periodSeconds: 5
            # One second longer than command timeout should prevent generation of zombie processes.
            timeoutSeconds: 6
            successThreshold: 1
            failureThreshold: 5
            exec:
              command:
                - sh
                - -c
                - /health/ping_liveness_local.sh 5
          readinessProbe:
            initialDelaySeconds: 5
            periodSeconds: 5
            timeoutSeconds: 2
            successThreshold: 1
            failureThreshold: 5
            exec:
              command:
                - sh
                - -c
                - /health/ping_readiness_local.sh 1
          resources:
            limits: {}
            requests: {}
          volumeMounts:
            - name: start-scripts
              mountPath: /opt/bitnami/scripts/start-scripts
            - name: health
              mountPath: /health
            - name: redis-data
              mountPath: /data
              subPath: 
            - name: config
              mountPath: /opt/bitnami/redis/mounted-etc
            - name: redis-tmp-conf
              mountPath: /opt/bitnami/redis/etc/
            - name: tmp
              mountPath: /tmp
      volumes:
        - name: start-scripts
          configMap:
            name: pipelines-exporter-redis-scripts
            defaultMode: 0755
        - name: health
          configMap:
            name: pipelines-exporter-redis-health
            defaultMode: 0755
        - name: config
          configMap:
            name: pipelines-exporter-redis-configuration
        - name: redis-tmp-conf
          emptyDir: {}
        - name: tmp
          emptyDir: {}
        - name: redis-data
          emptyDir: {}
---
# Source: gitlab-ci-pipelines-exporter/templates/servicemonitor.yaml
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: pipelines-exporter
  labels:
    app.kubernetes.io/name: gitlab-ci-pipelines-exporter
    app.kubernetes.io/version: v0.5.1
    helm.sh/chart: gitlab-ci-pipelines-exporter-0.2.6
    app.kubernetes.io/instance: pipelines-exporter
    app.kubernetes.io/managed-by: Helm

spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: gitlab-ci-pipelines-exporter
      app.kubernetes.io/version: v0.5.1
      helm.sh/chart: gitlab-ci-pipelines-exporter-0.2.6
      app.kubernetes.io/instance: pipelines-exporter
      app.kubernetes.io/managed-by: Helm
      helm.sh/from: deploy.pipelines-exporter
  endpoints:
    - port: http
      interval: 10s

When I try to use some custom health check there, it does not replace default one with httpGet, just added to it. I think maybe this is related to using "with" instruction of Helm somehow, maybe you can try to replace it with "if" which checks non-emptiness of that variable and then just "toYaml" like the following:

        {{- if .Values.livenessProbe }}
        livenessProbe:
          {{- toYaml .Values.livenessProbe | nindent 10 }}
        {{- end }}
        {{- if .Values.readinessProbe }}
        readinessProbe:
          {{- toYaml .Values.readinessProbe | nindent 10 }}
        {{- end }}

[mvisonneau]

Interesting, have you also looked into https://helm.sh/docs/chart_template_guide/values_files/#deleting-a-default-key ?

Doing something like helm upgrade [..] --set livenessProbe=null?

[caermeglaeddyv]

Yes, sure, I knew about that, forgot because not using often. I just think that it's more comfortable for users just to use empty map for that probe. And what about setting completely different values for probes? In your case, if I do that, my values will be added to existing ones. I need also to override them via "--set ..." and then after that I need to use my values file via "-f ..."? What if I use declarative only way to manage helm releases, for example some GitOps tool like GitOps Toolkit (Flux v2) ?

[mvisonneau]

Yep, not disagreeing here! I am no helm chart guru so will let anyone contribute to making this config more flexible :grimacing: .

[shivjm]

I think maybe this is related to using "with" instruction of Helm somehow, maybe you can try to replace it with "if" which checks non-emptiness of that variable and then just "toYaml" like the following:

with is shorthand for if that also makes . refer to the object within the block. What’s happening in this case is that Helm is merging the readinessProbe or livenessProbe in the defaults with the provided values (rather than replacing the defaults) before they ever reach the template. Maybe the solution is to move the defaults into the template, like this:

{{- with .Values.livenessProbe }}
          livenessProbe: {{ toYaml . | nindent 16 }}
{{- else }}
          livenessProbe:
            httpGet:
              path: /health/live
              port: 8080
{{- end }}

The only caveat is that I believe it would no longer be possible to turn off the health checks, but that seems unimportant.