How configure Vector to export Prometheus metrics

[nlamirault]

Hi, I configure the chart like that :

  role: Agent

  podMonitor:
    enabled: true

  customConfig:
    data_dir: /vector-data-dir
    api:
      enabled: true
      address: 127.0.0.1:8686
      playground: false

    sources:
      internal:
        type: internal_metrics
        scrape_interval_secs: 30
        namespace: logging
      kubernetes:
        type: kubernetes_logs

PodMonitor is generated like that :

✖ kubectl -n logging get podmonitor vector -o yaml
apiVersion: monitoring.coreos.com/v1
kind: PodMonitor
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"monitoring.coreos.com/v1","kind":"PodMonitor","metadata":{"annotations":{},"labels":{"app.kubernetes.io/component":"Agent","app.kubernetes.io/instance":"vector","app.kubernetes.io/managed-by":"Helm","app.kubernetes.io/name":"vector","app.kubernetes.io/version":"0.21.2-distroless-libc","argocd.argo
proj.io/instance":"vector","helm.sh/chart":"vector-0.10.3","monitoring":"portefaix","portefaix.xyz/version":"v0.35.0"},"name":"vector","namespace":"logging"},"spec":{"jobLabel":"app.kubernetes.io/name","namespaceSelector":{"matchNames":["logging"]},"podMetricsEndpoints":[{"honorLabels":false,"honorTimestamps":true,"pa
th":"/metrics","port":"prom-exporter"}],"selector":{"matchLabels":{"app.kubernetes.io/component":"Agent","app.kubernetes.io/instance":"vector","app.kubernetes.io/name":"vector"}}}}
  creationTimestamp: "2022-04-05T14:18:36Z"
  generation: 1
  labels:
    app.kubernetes.io/component: Agent
    app.kubernetes.io/instance: vector
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: vector
    app.kubernetes.io/version: 0.21.2-distroless-libc
    argocd.argoproj.io/instance: vector
    helm.sh/chart: vector-0.10.3
    monitoring: portefaix
    portefaix.xyz/version: v0.35.0
  name: vector
  namespace: logging
  resourceVersion: "4681778"
  uid: b4ba8696-d029-44af-9586-c81f5dea72b3
spec:
  jobLabel: app.kubernetes.io/name
  namespaceSelector:
    matchNames:
    - logging
  podMetricsEndpoints:
  - honorLabels: false
    honorTimestamps: true
    path: /metrics
    port: prom-exporter
  selector:
    matchLabels:
      app.kubernetes.io/component: Agent
      app.kubernetes.io/instance: vector
      app.kubernetes.io/name: vector

but the daemonset does not have the metrics port :

    name: vector
        ports:
        - containerPort: 8686
          name: api
          protocol: TCP
        resources:
          limits:
            cpu: 100m
            memory: 256Mi
          requests:
            cpu: 50m
            memory: 64Mi

Any idea how can i configure the Helm chart to add the Prometheus metrics port ? Thanks.

[juris]

Looks like if you are using customConfig, you have to manually define host metrics source and prometheus sink:

customConfig:
  sources:
    host-metrics:
      type: host_metrics
      scrape_interval_secs: 10
      filesystem:
        devices:
          excludes: ["binfmt_misc"]
  sinks:
    prom-exporter:
      type: prometheus_exporter
      inputs:
        - host-metrics
      address: 0.0.0.0:9598

Too bad it's missing in the documentation and you need to dig through helm helpers code.

[jszwedko]

What @juris mentioned is correct. If you are providing your own custom config, you have to wire up the prom exporter yourself. I'll leave this open to improve the documentation.

[nlamirault]

Thanks, it works fine.

[nikolasj]

@jszwedko Hello. Please tell me. Is there documentation on how to set up vector monitoring if I use helm chart + customconfig?

[vainkop]

@juris This is wrong. You should use.

  sources:
    vector_metrics:
      type: internal_metrics
      scrape_interval_secs: 10

  sinks:
    prom-exporter:
      type: prometheus_exporter
      inputs:
        - vector_metrics
      address: 0.0.0.0:9598

podMonitor:
  # podMonitor.enabled -- If true, create a PodMonitor for Vector.
  enabled: true

Vector docs

Helm chart values.yaml