joe-elliott / cert-exporter

A Prometheus exporter that publishes cert expirations on disk and in Kubernetes secrets
Apache License 2.0
313 stars 88 forks source link

cert-exporter doesn't provide metrics #141

Closed ep4sh closed 11 months ago

ep4sh commented 11 months ago

Good day, could you please advice where I am did wrong with cert-exporter installation?

I am usign helm chart installation (3.4.1) with mostrly default values:

certManager:
  # DaemonSet or Deployment
  kind: Deployment
  replicaCount: 1
  # Adds additional labels to pods
  additionalPodLabels: {}
  # label1: test
  # label2: test

  image:
    repository: joeelliott/cert-exporter
    # The default tag is ".Chart.AppVersion", only set "tag" to override that
    tag: 
    pullPolicy: IfNotPresent
    command: ["./app"]
    args:
      - --secrets-annotation-selector=cert-manager.io/certificate-name
      - --secrets-include-glob=*.crt
      - --logtostderr
  imagePullSecrets: []
  nameOverride: ""
  fullnameOverride: ""

  podAnnotations:
    prometheus.io/scrape: "true"
    prometheus.io/port: "8080"
    prometheus.io/path: "/metrics"

  podSecurityContext: {}
  # fsGroup: 2000

  securityContext: {}
    # capabilities:
    #   drop:s
    #   - ALL
    # readOnlyRootFilesystem: true
    # runAsNonRoot: true
    # runAsUser: 1000

  resources: 
    limits:
      cpu: 300m
      memory: 256Mi
    requests:
      cpu: 50m
      memory: 64Mi

  nodeSelector: {}

  tolerations: []

  affinity: {}

  volumes: []
    # - name: kubelet
    #   hostPath:
    #     path: /var/lib/kubelet
    #     type: Directory
  volumeMounts: []
    # - mountPath: /var/lib/kubelet/pki
    #   mountPropagation: HostToContainer
    #   name: kubelet
    #   readOnly: true

service:
  type: ClusterIP
  port: 8080

  portName: http-metrics

  # Annotations to add to the service
  annotations: {}

  # Requires prometheus-operator to be installed
  serviceMonitor:
    create: false

    # cannot be empty
    additionalLabels:
      prometheus.io/load-rule: "true"

    ## Scrape interval. If not set, the Prometheus default scrape interval is used.
    ##
    interval: 20s

    ## metric relabel configs to apply to samples before ingestion.
    ##
    metricRelabelings: []
    # - action: keep
    #   regex: 'kube_(daemonset|deployment|pod|namespace|node|statefulset).+'
    #   sourceLabels: [__name__]

    # relabel configs to apply to samples before ingestion.
    ##
    relabelings: []
    # - sourceLabels: [__meta_kubernetes_pod_node_name]
    #   separator: ;
    #   regex: ^(.*)$
    #   targetLabel: nodename
    #   replacement: $1
    #   action: replace

rbac:
  serviceAccount:
    # Specifies whether a service account should be created
    create: true
    # Annotations to add to the service account
    annotations: {}
    # The name of the service account to use.
    # If not set and create is true, a name is generated using the fullname template
    name: cert-exporter

  clusterRole:
    # New role to grant to the service account
    create: true
    # Annotations to add to the service account
    annotations: {}
    # Rules for the Role
    rules:
    - apiGroups: [""]
      resources: ["secrets"]
      verbs: ["get", "list"]

  clusterRoleBinding:
    create: true

dashboards:
  # Labels to add to all dashboard ConfigMaps
  additionalLabels:
    grafana_dashboard: "1"
  certManagerDashboard:
    create: false
  namespace: victoriametrics

I checked the application log and it cannot determine its own version:

I0927 12:28:51.974958       1 main.go:97] Starting cert-exporter (version unknown; commit unknown; date unknown)
W0927 12:28:51.975162       1 client_config.go:617] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I0927 12:28:51.975548       1 periodicSecretChecker.go:65] Begin periodic check
I0927 12:28:54.374275       1 periodicSecretChecker.go:115] Reviewing secret default-token-srr4n in allure
....
.....

But as I understood it was started as expected - at least I can see my k8s secrets with TLS certificates which I want to monitor for expiration:

....
I0927 12:28:54.376133       1 periodicSecretChecker.go:115] Reviewing secret ttl-controller-token-8l2wg in kube-system
I0927 12:28:54.376142       1 periodicSecretChecker.go:115] Reviewing secret waf-ingress-admission in kube-system
I0927 12:28:54.376147       1 periodicSecretChecker.go:115] Reviewing secret waf-ingress-backend-token-cfbpw in kube-system
I0927 12:28:54.376152       1 periodicSecretChecker.go:115] Reviewing secret waf-ingress-default-tls in kube-system
I0927 12:28:54.376161       1 periodicSecretChecker.go:115] Reviewing secret waf-ingress-secret in kube-system
I0927 12:28:54.376167       1 periodicSecretChecker.go:115] Reviewing secret waf-ingress-token-7bcmw in kube-system
....

When I apply a dashboard to the Grafana, I don't see any info about the desired certificates: 2023-09-27_16-06

Moreover, I checked the pod's metrics and there only 1 (OMG!) metric related to the expiration: 2023-09-27_15-56

Do you have ideas how to fix it? It might be I missed some tricky flag?

Thanks in advance, Pasha

ep4sh commented 11 months ago

btw I don't see this line in my logs: https://github.com/joe-elliott/cert-exporter/blob/master/src/checkers/periodicSecretChecker.go#L164

ep4sh commented 11 months ago

Got it: the default helm chart contains analyze only for annotated secrets - that I missed. in other words I deleted:

      - --secrets-annotation-selector=cert-manager.io/certificate-name
joe-elliott commented 11 months ago

i appreciate you sticking with this. i didn't write the helm charts and only update them when i cut a new version of cert-exporter.