fluent / fluent-bit

Fast and Lightweight Logs and Metrics processor for Linux, BSD, OSX and Windows
https://fluentbit.io
Apache License 2.0
5.56k stars 1.53k forks source link

prometheus_scrape: parsing bug on negative gauge values #8970

Open diseq opened 1 month ago

diseq commented 1 month ago

Bug Report

Describe the bug Negative values returned from a prometheus exporter result in a parsed value of zero and an incorrect datetime assigned. The source used is the openwrt node exporter: https://github.com/openwrt/packages/tree/openwrt-22.03/utils/prometheus-node-exporter-lua

Prometheus return:

# TYPE wifi_network_noise_dbm gauge
# TYPE wifi_network_signal_dbm gauge
# TYPE wifi_network_quality gauge
wifi_network_noise_dbm{mode="Master",ifname="phy0-ap0",ssid="abc",channel="36",device="radio0",bssid="f4:f2:6d:aa:aa:aa",country="US",frequency="5180"} -106
wifi_network_signal_dbm{mode="Master",ifname="phy0-ap0",ssid="abc",channel="36",device="radio0",bssid="f4:f2:6d:aa:aa:aa",country="US",frequency="5180"} -255
wifi_network_quality{mode="Master",ifname="phy0-ap0",ssid="abc",channel="36",device="radio0",bssid="f4:f2:6d:aa:aa:aa",country="US",frequency="5180"} 2

fluent-bit parsed return

1970-01-01T00:00:00.000000000Z wifi_network_noise_dbm{mode="Master",ifname="phy0-ap0",ssid="abc",channel="36",device="radio0",bssid="f4:f2:6d:aa:aa:aa",country="US",frequency="5180"} = 0
1970-01-01T00:00:00.000000000Z wifi_network_signal_dbm{mode="Master",ifname="phy0-ap0",ssid="abc",channel="36",device="radio0",bssid="f4:f2:6d:aa:aa:aa",country="US",frequency="5180"} = 0
2024-06-17T05:00:12.059322400Z wifi_network_quality{mode="Master",ifname="phy0-ap0",ssid="abc",channel="36",device="radio0",bssid="f4:f2:6d:aa:aa:aa",country="US",frequency="5180"} = 2

Based on the prometheus exposition formats documentation (https://github.com/prometheus/docs/blob/main/content/docs/instrumenting/exposition_formats.md), value is defined as double:

value is a float represented as required by Go's [ParseFloat()](https://golang.org/pkg/strconv/#ParseFloat) function. In addition to standard numerical values, NaN, +Inf, and -Inf are valid values representing not a number, positive infinity, and negative infinity, respectively.

To Reproduce

run fluent-bit with the following config (--config). metrics path is just the encoded return example from above.

service:
  flush: 1
  log_level: warn
  http_server: off
  daemon: off

pipeline:
  inputs:
    - name: prometheus_scrape
      tag: metrics.prom
      host: httpbin.org
      tls: on
      port: 443
      metrics_path: /base64/IyBUWVBFIHdpZmlfbmV0d29ya19ub2lzZV9kYm0gZ2F1Z2UNCiMgVFlQRSB3aWZpX25ldHdvcmtfc2lnbmFsX2RibSBnYXVnZQ0KIyBUWVBFIHdpZmlfbmV0d29ya19xdWFsaXR5IGdhdWdlDQp3aWZpX25ldHdvcmtfbm9pc2VfZGJte21vZGU9Ik1hc3RlciIsaWZuYW1lPSJwaHkwLWFwMCIsc3NpZD0iYWJjIixjaGFubmVsPSIzNiIsZGV2aWNlPSJyYWRpbzAiLGJzc2lkPSJmNDpmMjo2ZDphYTphYTphYSIsY291bnRyeT0iVVMiLGZyZXF1ZW5jeT0iNTE4MCJ9IC0xMDYNCndpZmlfbmV0d29ya19zaWduYWxfZGJte21vZGU9Ik1hc3RlciIsaWZuYW1lPSJwaHkwLWFwMCIsc3NpZD0iYWJjIixjaGFubmVsPSIzNiIsZGV2aWNlPSJyYWRpbzAiLGJzc2lkPSJmNDpmMjo2ZDphYTphYTphYSIsY291bnRyeT0iVVMiLGZyZXF1ZW5jeT0iNTE4MCJ9IC0yNTUNCndpZmlfbmV0d29ya19xdWFsaXR5e21vZGU9Ik1hc3RlciIsaWZuYW1lPSJwaHkwLWFwMCIsc3NpZD0iYWJjIixjaGFubmVsPSIzNiIsZGV2aWNlPSJyYWRpbzAiLGJzc2lkPSJmNDpmMjo2ZDphYTphYTphYSIsY291bnRyeT0iVVMiLGZyZXF1ZW5jeT0iNTE4MCJ9IDI=
      scrape_interval: 2

  outputs:
    - name: stdout
      match: metrics.prom

Expected behavior

correctly parsed values:

2024-06-17T05:00:12.059322400Z wifi_network_noise_dbm{mode="Master",ifname="phy0-ap0",ssid="abc",channel="36",device="radio0",bssid="f4:f2:6d:aa:aa:aa",country="US",frequency="5180"} = -106
2024-06-17T05:00:12.059322400Z wifi_network_signal_dbm{mode="Master",ifname="phy0-ap0",ssid="abc",channel="36",device="radio0",bssid="f4:f2:6d:aa:aa:aa",country="US",frequency="5180"} = -255
2024-06-17T05:00:12.059322400Z wifi_network_quality{mode="Master",ifname="phy0-ap0",ssid="abc",channel="36",device="radio0",bssid="f4:f2:6d:aa:aa:aa",country="US",frequency="5180"} = 2

Your Environment

edsiper commented 2 weeks ago

moving it to the next milestone