search

prometheus-community / helm-charts

Prometheus community Helm charts
Apache License 2.0
5k stars 4.99k forks source link

[prometheus-pushgateway] Prometheus can't scrape data from Pushgateway when using basic authentication #4821

Open FerasMaali opened 3 weeks ago

FerasMaali commented 3 weeks ago

Describe the bug a clear and concise description of what the bug is.

Prometheus received a 401 status code when scraping data from PushGateway after enabling HTTP basic authentication. The sericemonitor resource used to define the PushGateway as a target for Prometheus is created by the chart itself by setting serviceMonitor.enabled to true.

What's your helm version?

version.BuildInfo{Version:"v3.15.4", GitCommit:"fa9efb07d9d8debbb4306d72af76a383895aa8c4", GitTreeState:"clean", GoVersion:"go1.22.6"}

What's your kubectl version?

Client Version: v1.29.8 Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3 Server Version: v1.28.11-eks-db838b0

Which chart?

prometheus-pushgateway

What's the chart version?

prometheus-pushgateway-2.14.0

What happened?

No response

What you expected to happen?

Prometheus should be able to scrape data without any issues

How to reproduce it?

No response

Enter the changed values of values.yaml?

webConfiguration:
  basicAuthUsers:
    username: password

replicaCount: 1

serviceMonitor:
  enabled: true
  interval: 60s
  additionalLabels:
    release: my-prometheus

Enter the command that you execute and failing/misfunctioning.

curl http://<prometheus_server>/api/v1/targets | jq -r '
    .data.activeTargets[] |
    select(.discoveredLabels.__meta_kubernetes_endpoints_label_app_kubernetes_io_name == "prometheus-pushgateway") |
    .lastError
'

Anything else we need to know?

No response

zeritti commented 3 weeks ago

Prometheus received a 401 status code when scraping data from PushGateway after enabling HTTP basic authentication.

If you enable basic auth at Pushgateway, Prometheus must authenticate itself when scraping it, i.e. you have to tell Prometheus of the username and password.

The sericemonitor resource used to define the PushGateway as a target for Prometheus is created by the chart itself by setting serviceMonitor.enabled to true.

The service monitor must include basic auth configuration (username, password). The prometheus-pushgateway chart's service monitor does not currently support basic auth (field servicemonitor.spec.endpoints.basicAuth), though (a PR will be appreciated). At the moment, you'd have to create your own service monitor.