prometheus / blackbox_exporter

Blackbox prober exporter
https://prometheus.io
Apache License 2.0
4.7k stars 1.05k forks source link

http's probe - API - cookie - authenticate #939

Closed DaBeOps closed 2 years ago

DaBeOps commented 2 years ago

Host operating system: output of uname -a

docker: 20.10.14 ubuntu server: 20.04

blackbox_exporter version: output of blackbox_exporter --version

v0.19.0

What is the blackbox.yml module config.

  http_pbs-api_GET_datastore-usage:
    prober: 'http'
    http:
      valid_http_versions: ["HTTP/1.1", "HTTP/2.0"]
      method: 'GET'
      fail_if_not_ssl: true
      headers:
        Content-Type: application/json
        PBSAuthCookie: ***

What is the prometheus.yml scrape config.

  - job_name: 'blackbox_http_pbs-api_GET_datastore-usage'
    metrics_path: '/probe'
    params:
      module:
        - 'http_pbs-api_GET_datastore-usage'
    static_configs:
      - targets:
          - 'https://[FQDN]:8007/api2/json/status/datastore-usage'
    relabel_configs:
      - source_labels: ['__address__']
        target_label: '__param_target'
      - source_labels: ['__param_target']
        target_label: 'instance'
      - target_label: '__address__'
        replacement: 'blackbox_exporter:9115'

What logging output did you get from adding &debug=true to the probe URL?

Logs for the probe:
ts=2022-06-29T11:50:43.009973288Z caller=main.go:320 module=http_pbs-api_GET_datastore-usage target=https://[FQDN]:8007/api2/json/status/datastore-usage level=info msg="Beginning probe" probe=http timeout_seconds=9.5
ts=2022-06-29T11:50:43.01010887Z caller=http.go:335 module=http_pbs-api_GET_datastore-usage target=https://[FQDN]:8007/api2/json/status/datastore-usage level=info msg="Resolving target address" ip_protocol=ip6
ts=2022-06-29T11:50:43.011592914Z caller=http.go:335 module=http_pbs-api_GET_datastore-usage target=https://[FQDN]:8007/api2/json/status/datastore-usage level=info msg="Resolving target address" ip_protocol=ip4
ts=2022-06-29T11:50:43.012733298Z caller=http.go:335 module=http_pbs-api_GET_datastore-usage target=https://[FQDN]:8007/api2/json/status/datastore-usage level=info msg="Resolved target address" ip=****
ts=2022-06-29T11:50:43.012812737Z caller=client.go:251 module=http_pbs-api_GET_datastore-usage target=https://[FQDN]:8007/api2/json/status/datastore-usage level=info msg="Making HTTP request" url=https://[IP]:8007/api2/json/status/datastore-usage host=[FQDN]:8007
ts=2022-06-29T11:50:46.028874375Z caller=main.go:130 module=http_pbs-api_GET_datastore-usage target=https://[FQDN]:8007/api2/json/status/datastore-usage level=info msg="Received HTTP response" status_code=401
ts=2022-06-29T11:50:46.028944435Z caller=main.go:130 module=http_pbs-api_GET_datastore-usage target=https://[FQDN]:8007/api2/json/status/datastore-usage level=info msg="Invalid HTTP response status code, wanted 2xx" status_code=401
ts=2022-06-29T11:50:46.029089896Z caller=main.go:130 module=http_pbs-api_GET_datastore-usage target=https://[FQDN]:8007/api2/json/status/datastore-usage level=info msg="Response timings for roundtrip" roundtrip=0 start=2022-06-29T13:50:43.012875263+02:00 dnsDone=2022-06-29T13:50:43.012875263+02:00 connectDone=2022-06-29T13:50:43.016534377+02:00 gotConn=2022-06-29T13:50:43.022949955+02:00 responseStart=2022-06-29T13:50:46.028805306+02:00 tlsStart=2022-06-29T13:50:43.01655198+02:00 tlsDone=2022-06-29T13:50:43.022937071+02:00 end=2022-06-29T13:50:46.029071051+02:00
ts=2022-06-29T11:50:46.029164384Z caller=main.go:320 module=http_pbs-api_GET_datastore-usage target=https://[FQDN]:8007/api2/json/status/datastore-usage level=error msg="Probe failed" duration_seconds=3.019147575

What did you expect to see?

Below is an example of an answer from the API.

{
  "data": [
    {
      "avail": 526646460416,
      "estimated-full-date": 1684642366,
      "gc-status": {
        "disk-bytes": 333086373407,
        "disk-chunks": 127054,
        "index-data-bytes": 1555189549120,
        "index-file-count": 6,
        "pending-bytes": 22174066350,
        "pending-chunks": 7607,
        "removed-bad": 0,
        "removed-bytes": 23560664542,
        "removed-chunks": 7722,
        "still-bad": 0,
        "upid": "UPID:pbs:0000028B:000001BA:00002A84:62BB79E0:garbage_collection:DS\\x2dMail:root@pam:"
      },
      "history": [
        null,
        0.7911734438266355,
        ...
      ]
    }
  ]
}

This exemple are got with this command:

curl -vvv --silent --insecure --cookie "PBSAuthCookie=$TICKET" \
  --request  GET https://${FQDN}:8007/api2/json/status/datastore-usage | jq '.'

What did you see instead?

I'm using Proxmox Backup Server and I try to catch some importants informations using the API. For exemple I would know if the datastore is full (.data[].avail) To fetech this information I have to authenticate using a cookie. Basic authentification(username/password) doesn't work. PBS(Proxmox Backup Server) are waiting for a cookie with this parameter: PBSAuthCookie

My question is: is it possible to get an equivalent to the curl command below ?

dgl commented 2 years ago

blackbox_exporter isn't suited to this, try https://github.com/prometheus-community/json_exporter instead.