Node=1&Cluster=0 Returns no data

[znachtman]

When I access

http://pve-exporter:9221/pve?target=nodename.foo&node=1&cluster=0

I only get:

# HELP pve_onboot_status Proxmox vm config onboot value
# TYPE pve_onboot_status gauge

Additional information:

• I'm using prompve/prometheus-pve-exporter:latest (I've also tried other 3.x images).
• I've added the read-only proxmox user as described (in my case: pve-prometheus-ro).
• I've also confirmed that accessing proxmox with the read-only user and manually accessing the API returns information (e.g. https://nodename.foo:8006/api2/json/nodes/nodename/qemu/133/config) correctly returns information. I can do this across my proxmox nodes.
• I've confirmed via proxmox logs that the PVE exporter appears to be calling /nodes/ APIs and getting back correct payloads (via tail -n 400 /var/log/pveproxy/access.log on one of the proxmox nodes). See below for an example of the access logs on the PVE node...

::ffff:172.16.10.9 - pve-prometheus-ro@pve [09/01/2024:22:43:56 -0800] "GET /api2/json/nodes/nodename/qemu/112/config HTTP/1.1" 200 562
::ffff:172.16.10.9 - pve-prometheus-ro@pve [09/01/2024:22:43:56 -0800] "GET /api2/json/nodes/nodename/qemu/133/config HTTP/1.1" 200 437
::ffff:172.16.10.9 - pve-prometheus-ro@pve [09/01/2024:22:43:56 -0800] "GET /api2/json/nodes/nodename/qemu/103/config HTTP/1.1" 200 647
::ffff:172.16.10.9 - pve-prometheus-ro@pve [09/01/2024:22:43:56 -0800] "GET /api2/json/nodes/nodename/qemu/131/config HTTP/1.1" 200 433
::ffff:172.16.10.9 - pve-prometheus-ro@pve [09/01/2024:22:43:56 -0800] "GET /api2/json/nodes/nodename/qemu/109/config HTTP/1.1" 200 434
::ffff:172.16.10.9 - pve-prometheus-ro@pve [09/01/2024:22:43:56 -0800] "GET /api2/json/nodes/nodename/qemu/101/config HTTP/1.1" 200 433
::ffff:172.16.10.9 - pve-prometheus-ro@pve [09/01/2024:22:43:56 -0800] "GET /api2/json/nodes/nodename/qemu/100/config HTTP/1.1" 200 527

Nothing is in the prompve/prometheus-pve-exporter logs, outside of TLS warnings that are generated when accessing various proxmox nodes (e.g. /opt/prometheus-pve-exporter/lib/python3.11/site-packages/urllib3/connectionpool.py:1099: InsecureRequestWarning: Unverified HTTPS request is being made to host...).

I'm able to access cluster metrics just fine.

It appears the APIs are getting queried, data is correctly getting returned to PVE exporter (from proxmox); however, it looks like that data is not being returned from PVE exporter as far as I can tell.

[znachtman]

Also for reference, I've reviewed this item in detail: https://github.com/prometheus-pve/prometheus-pve-exporter/issues/156.

I'm also using the grafana dashboard referenced in the readme https://grafana.com/grafana/dashboards/10347-proxmox-via-prometheus/. At the moment, I'm getting a lot of duplicate data, as each node is returning cluster wide metrics. I'm hoping converting the queries to cluster=0&node=1 fixes this issue on the dashboard side, if I can get query parameters to return node-level data.

[znerol]

This is expected behavior. The only node metric is pve_onboot_status. All other metrics are cluster metrics.