My Prometheus is unable to scrape any telemetry data from gNMIc although gNMIc is collecting data from the target and exposing it on port 9273.

[arbabali999]

PFB prometheus.yml and gnmic.yml. My Prometheus is unable to scrape any telemetry data from gnmic although gnmic is collecting data from the target and exposing it on port 9273.

prometheus.yml:

scrape_configs:

• job_name: "Prometheus" static_configs:

  • targets: ["localhost:9090"]

• job_name: "gnmic" static_configs:

  • targets: ["localhost:9273"]

• job_name: "Grafana" static_configs:

  • targets: ["localhost:3000"]

gnmic.yml:

port: 57400 timeout: 10s insecure: true

targets: 192.168.69.38: username: admin password: admin123 subscriptions:

• sros_port_stats
• sros_ce_port_stats
• sros_cpu
• sros_memory_pool
• sros_anysec_stats

subscriptions: sros_cpu: paths:

• "/state/system/cpu[sample-period=60]/summary/usage" mode: stream stream-mode: sample sample-interval: 5s

  sros_memory_pool: paths:

• "/state/system/memory-pools" mode: stream stream-mode: sample sample-interval: 5s

  sros_port_stats: paths:

• "/state/port[port-id=1/1/c2/1]/statistics"
• "/state/port[port-id=1/1/c3/1]/statistics"
• "/state/port[port-id=1/1/c33/1]/statistics"
• "/state/port[port-id=esat-1/1/1]/statistics"
• "/state/port[port-id=esat-1/1/3]/statistics"
• "/state/port[port-id=esat-1/1/4]/statistics"
• "/state/port[port-id=esat-1/1/5]/statistics"
• "/state/port[port-id=esat-1/1/6]/statistics"

• "/state/port[port-id=esat-1/1/8]/statistics" mode: stream stream-mode: sample sample-interval: 5s

  sros_ce_port_stats: paths:

• "/state/port[port-id=1/1/c27/1]/statistics"
• "/state/port[port-id=1/1/c28/1]/statistics"
• "/state/port[port-id=1/1/c29/1]/statistics" mode: stream stream-mode: sample sample-interval: 5s

outputs: prom: type: prometheus listen: :9273 path: /metrics metric-prefix: gnmic append-subscription-name: true export-timestamps: true debug: false event-processors:

• trim-prefixes
• esa-health

processors: GNU nano 6.2
trim-prefixes: event-strings: value-names:

• ".*" transforms:

• path-base: apply-on: "name"

  esa-health: event-strings: value-names:

• hardware-status
• power-supply-1-status
• power-supply-2-status
• fan-status
• temperature-status transforms:
• replace: apply-on: "value" old: "ok" new: "1"
• replace: apply-on: "value" old: "degraded" new: "0"
• replace: apply-on: "value" old: "unknown" new: "2"
• replace: apply-on: "value" old: "failed" new: "3"

I have verified using below command (providing one as reference) about subscription path provided in gnmic.yml that subscriptions are correct.

sudo gnmic -a 192.168.69.38:57400 --insecure -u admin -p admin123 -d sub --path /state/system/cpu[sample-period=60]/summary/usage

Manual Output:

"source": "192.168.69.38:57400", "subscription-name": "default-1710885725", "timestamp": 1017826625194935505, "time": "2002-04-03T14:37:05.194935505+05:00", "prefix": "state/system/cpu[sample-period=60]/summary/usage", "updates": [ { "Path": "cpu-time", "values": { "cpu-time": 47593972 } }, { "Path": "time-used", "values": { "time-used": "47593972" } }, { "Path": "cpu-usage", "values": { "cpu-usage": "4.97" } } ] }

Same can be verified using, sudo journalctl -u gnmic.service -f as it is showing the results on real time.

Also, i don't get anything when using below command.

curl http://localhost:9273/metrics

[karimra]

"timestamp": 1017826625194935505,
"time": "2002-04-03T14:37:05.194935505+05:00",

Your device is reporting old timestamps (probably not synced to NTP). Metrics expire if their timestamp is older that 60s. Either fix the timestamps reported by the device or set expiration: -1 under the prometheus output to avoid metrics being deleted. The second solution means that metrics will never go stale and expire which might not be what you want.

[arbabali999]

It's my testing router so i performed second solution. So, i have started to see the information using curl http://localhost:9273/metrics but Prometheus is still unable to reflect anything using expression bar. Can you please assist.

gnmic-scrap-log.txt

[arbabali999]

Thanks alot. Issue resolved. expiration: -1 only started showing metrics in the browser. Eventually i had to sync the time of both Router and gNMIc. Prometheus then started scrapping metrics.

[karimra]

Good to hear. You probably have export-timestamps: true, in that case gNMIc exports the timestamp of the metric to Prometheus. That timestamp being too far in the past, is dropped by Prometheus. If you change export-timestmaps to false, the first solution would have been enough.