search

karimra / gnmic

gNMIc is a gNMI CLI client and collector
https://gnmic.kmrd.dev
Apache License 2.0
217 stars 32 forks source link

strings-as-labels does not work for prometheus remote-write #646

Open Sparc0 opened 2 years ago

Sparc0 commented 2 years ago

It seems that the strings-as-labels config variable is not working on the prometheus-remote-write output. It works for the prometheus scrape based output.

Paths i subscribed on to make sure i get one with a string value and one with a int value. 

"/components/component[name=FPC5]/state/software-version"
"/components/component[name=FPC5]/state/allocated-power"

stdout

[
  {
    "name": "interface_state",
    "timestamp": 1663687551088646387,
    "tags": {
      "component_name": "FPC5",
      "source": "junos-mx-960",
      "subscription-name": "interface_state"
    },
    "values": {
      "/components/component/state/software-version": "Version 19.4R3-S7.3 by builder on 2021-12-22 20:43:49 UTC"
    }
  }
]
[
  {
    "name": "interface_state",
    "timestamp": 1663687553101994654,
    "tags": {
      "component_name": "FPC5",
      "source": "junos-mx-960",
      "subscription-name": "interface_state"
    },
    "values": {
      "/components/component/state/allocated-power": 496
    }
  }
]

prometheus scrape based output

# HELP gnmic_components_component_state_allocated_power gNMIc generated metric
# TYPE gnmic_components_component_state_allocated_power untyped
gnmic_components_component_state_allocated_power{component_name="FPC5",source="junos-mx-960",subscription_name="interface_state"} 496
# HELP gnmic_components_component_state_software_version gNMIc generated metric
# TYPE gnmic_components_component_state_software_version untyped
gnmic_components_component_state_software_version{component_name="FPC5",software_version="Version 19.4R3-S7.3 by builder on 2021-12-22 20:43:49 UTC",source="junos-mx-960",subscription_name="interface_state"} 1

prometheus remote write debug log

2022/09/20 17:26:21.261548 [prometheus_write_output:prom-remote] got event to buffer: {"name":"interface_state","timestamp":1663687581256656375,"tags":{"component_name":"FPC5","source":"junos-mx-960","subscription-name":"interface_state"},"values":{"/components/component/state/software-version":"Version 19.4R3-S7.3 by builder on 2021-12-22 20:43:49 UTC"}}
[
  {
    "name": "interface_state",
    "timestamp": 1663687581256656375,
    "tags": {
      "component_name": "FPC5",
      "source": "junos-mx-960",
      "subscription-name": "interface_state"
    },
    "values": {
      "/components/component/state/software-version": "Version 19.4R3-S7.3 by builder on 2021-12-22 20:43:49 UTC"
    }
  }
]
2022/09/20 17:26:21.261830 [prometheus_output:output-prom] got event to store: {"name":"interface_state","timestamp":1663687581256656375,"tags":{"component_name":"FPC5","source":"junos-mx-960","subscription-name":"interface_state"},"values":{"/components/component/state/software-version":"Version 19.4R3-S7.3 by builder on 2021-12-22 20:43:49 UTC"}}
2022/09/20 17:26:21.261887 [prometheus_output:output-prom] saved key=18357390127090588070, metric: name=gnmic_components_component_state_software_version,labels=[component_name=FPC5,software_version=Version 19.4R3-S7.3 by builder on 2021-12-22 20:43:49 UTC,source=junos-mx-960,subscription_name=interface_state],value=1.000000,time=nil,addedAt=2022-09-20 17:2
6:21.261842349 +0200 CEST m=+38.509476886
[
  {
    "name": "interface_state",
    "timestamp": 1663687581263683429,
    "tags": {
      "component_name": "FPC5",
      "source": "junos-mx-960",
      "subscription-name": "interface_state"
    },
    "values": {
      "/components/component/state/allocated-power": 496
    }
  }
]
2022/09/20 17:26:21.466317 [prometheus_output:output-prom] got event to store: {"name":"interface_state","timestamp":1663687581263683429,"tags":{"component_name":"FPC5","source":"junos-mx-960","subscription-name":"interface_state"},"values":{"/components/component/state/allocated-power":496}}
2022/09/20 17:26:21.466386 [prometheus_output:output-prom] saved key=5079452650561576165, metric: name=gnmic_components_component_state_allocated_power,labels=[component_name=FPC5,source=junos-mx-960,subscription_name=interface_state],value=496.000000,time=nil,addedAt=2022-09-20 17:26:21.466330255 +0200 CEST m=+38.713964789
2022/09/20 17:26:21.466414 [prometheus_write_output:prom-remote] got event to buffer: {"name":"interface_state","timestamp":1663687581263683429,"tags":{"component_name":"FPC5","source":"junos-mx-960","subscription-name":"interface_state"},"values":{"/components/component/state/allocated-power":496}}
2022/09/20 17:26:21.466435 [prometheus_write_output:prom-remote] saving metrics metadata
2022/09/20 17:26:21.466439 [prometheus_write_output:prom-remote] writing TimeSeries to buffer
2022/09/20 17:26:24.153424 [prometheus_write_output:prom-remote] write interval reached, writing to remote
2022/09/20 17:26:24.153453 [prometheus_write_output:prom-remote] write triggered, buffer size: 1
2022/09/20 17:26:24.153501 [prometheus_write_output:prom-remote] writing a 1 time series chunk
2022/09/20 17:26:24.154284 [prometheus_write_output:prom-remote] got response from remote: status=204 No Content
2022/09/20 17:26:44.154559 [prometheus_write_output:prom-remote] write interval reached, writing to remote
2022/09/20 17:26:44.154590 [prometheus_write_output:prom-remote] write triggered, buffer size: 0
2022/09/20 17:26:44.154563 [prometheus_write_output:prom-remote] writing 1 metadata points
2022/09/20 17:26:44.155262 [prometheus_write_output:prom-remote] got response from remote: status=204 No Content

Telegraf debug log

2022-09-20T15:27:39Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2022-09-20T15:27:49Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2022-09-20T15:27:59Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
# HELP prometheus_remote_write_components_component_state_allocated_power Telegraf collected metric
# TYPE prometheus_remote_write_components_component_state_allocated_power untyped
prometheus_remote_write_components_component_state_allocated_power{component_name="FPC5",host="mon1",source="junos-mx-960",subscription_name="interface_state"} 496
2022-09-20T15:28:09Z D! [outputs.file] Wrote batch of 1 metrics in 96.394µs
2022-09-20T15:28:09Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2022-09-20T15:28:19Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
# HELP prometheus_remote_write_components_component_state_allocated_power Telegraf collected metric
# TYPE prometheus_remote_write_components_component_state_allocated_power untyped
prometheus_remote_write_components_component_state_allocated_power{component_name="FPC5",host="mon1",source="junos-mx-960",subscription_name="interface_state"} 496
2022-09-20T15:28:29Z D! [outputs.file] Wrote batch of 1 metrics in 123.301µs
2022-09-20T15:28:29Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2022-09-20T15:28:39Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
Sparc0 commented 2 years ago

I use version 0.26 of gnmic.

Sparc0 commented 2 years ago

647

pull request verified working in my env.

[
  {
    "name": "interface_state",
    "timestamp": 1663692516962334193,
    "tags": {
      "component_name": "FPC5",
      "source": "junos-mx-960",
      "subscription-name": "interface_state"
    },
    "values": {
      "/components/component/state/software-version": "Version 19.4R3-S7.3 by builder on 2021-12-22 20:43:49 UTC"
    }
  }
]
2022/09/20 18:48:36.967232 [prometheus_output:output-prom] got event to store: {"name":"interface_state","timestamp":1663692516962334193,"tags":{"component_name":"FPC5","source":"junos-mx-960","subscription-name":"interface_state"},"values":{"/components/component/state/software-version":"Version 19.4R3-S7.3 by builder on 2021-12-22 20:43:49 UTC"}}
2022/09/20 18:48:36.967293 [prometheus_output:output-prom] saved key=18357390127090588070, metric: name=gnmic_components_component_state_software_version,labels=[component_name=FPC5,software_version=Version 19.4R3-S7.3 by builder on 2021-12-22 20:43:49 UTC,source=junos-mx-960,subscription_name=interface_state],value=1.000000,time=nil,addedAt=2022-09-20 18:48:36.967248785 +0200 CEST m=+37.883707192
2022/09/20 18:48:36.967234 [prometheus_write_output:prom-remote] got event to buffer: {"name":"interface_state","timestamp":1663692516962334193,"tags":{"component_name":"FPC5","source":"junos-mx-960","subscription-name":"interface_state"},"values":{"/components/component/state/software-version":"Version 19.4R3-S7.3 by builder on 2021-12-22 20:43:49 UTC"}}
2022/09/20 18:48:36.967327 [prometheus_write_output:prom-remote] saving metrics metadata
2022/09/20 18:48:36.967331 [prometheus_write_output:prom-remote] writing TimeSeries to buffer
[
  {
    "name": "interface_state",
    "timestamp": 1663692516969377531,
    "tags": {
      "component_name": "FPC5",
      "source": "junos-mx-960",
      "subscription-name": "interface_state"
    },
    "values": {
      "/components/component/state/allocated-power": 496
    }
  }
]
2022/09/20 18:48:37.170235 [prometheus_output:output-prom] got event to store: {"name":"interface_state","timestamp":1663692516969377531,"tags":{"component_name":"FPC5","source":"junos-mx-960","subscription-name":"interface_state"},"values":{"/components/component/state/allocated-power":496}}
2022/09/20 18:48:37.170277 [prometheus_write_output:prom-remote] got event to buffer: {"name":"interface_state","timestamp":1663692516969377531,"tags":{"component_name":"FPC5","source":"junos-mx-960","subscription-name":"interface_state"},"values":{"/components/component/state/allocated-power":496}}
2022/09/20 18:48:37.170304 [prometheus_write_output:prom-remote] saving metrics metadata
2022/09/20 18:48:37.170308 [prometheus_write_output:prom-remote] writing TimeSeries to buffer
2022/09/20 18:48:37.170309 [prometheus_output:output-prom] saved key=5079452650561576165, metric: name=gnmic_components_component_state_allocated_power,labels=[component_name=FPC5,source=junos-mx-960,subscription_name=interface_state],value=496.000000,time=nil,addedAt=2022-09-20 18:48:37.170256367 +0200 CEST m=+38.086714778
2022/09/20 18:48:41.481888 [prometheus_write_output:prom-remote] write interval reached, writing to remote
2022/09/20 18:48:41.481913 [prometheus_write_output:prom-remote] write triggered, buffer size: 2
2022/09/20 18:48:41.481953 [prometheus_write_output:prom-remote] writing a 2 time series chunk

Telegraf debug log

2022-09-20T16:48:57Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2022-09-20T16:49:07Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2022-09-20T16:49:17Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
# HELP prometheus_remote_write_components_component_state_software_version Telegraf collected metric
# TYPE prometheus_remote_write_components_component_state_software_version untyped
prometheus_remote_write_components_component_state_software_version{component_name="FPC5",host="mon1",software_version="Version 19.4R3-S7.3 by builder on 2021-12-22 20:43:49 UTC",source="junos-mx-960",subscription_name="interface_state"} 1
# HELP prometheus_remote_write_components_component_state_allocated_power Telegraf collected metric
# TYPE prometheus_remote_write_components_component_state_allocated_power untyped
prometheus_remote_write_components_component_state_allocated_power{component_name="FPC5",host="mon1",source="junos-mx-960",subscription_name="interface_state"} 496
2022-09-20T16:49:27Z D! [outputs.file] Wrote batch of 2 metrics in 108.601µs
2022-09-20T16:49:27Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics