search

influxdata / telegraf

Agent for collecting, processing, aggregating, and writing metrics, logs, and other arbitrary data.
https://influxdata.com/telegraf
MIT License
14.68k stars 5.59k forks source link

GNMI / Got empty metric-name for response / #14044

Closed jorisd closed 1 year ago

jorisd commented 1 year ago

Relevant telegraf.conf

[[inputs.gnmi]]
  addresses = ['${addresses}']
  username = "xxx"
  password = "xxx"
  redial = "60s"
  encoding = "json_ietf"
  [[inputs.gnmi.subscription]]
    name = "ifdesc"
    origin = "openconfig-interfaces"
    path = '/interfaces/interface[name=FourHundredGigE*]/state/description'
    subscription_mode = "sample"
    sample_interval = "60s"
  [[inputs.gnmi.subscription]]
    name = "temperature"
    origin = "openconfig-platform"
    path = '/components/component[name=FourHundredGigE*]/state/temperature/avg'
    subscription_mode = "target_defined"
  [[inputs.gnmi.subscription]]
    name = "Cisco-optics-lanes"
    origin = "Cisco-IOS-XR-controller-optics-oper"
    path = '/optics-oper/optics-ports/optics-port[name=Optics0/2/0/*]/optics-lanes/optics-lane'
    subscription_mode = "target_defined"
  [[inputs.gnmi.subscription]]
    name = "Cisco-optics-info"
    origin = "Cisco-IOS-XR-controller-optics-oper"
    path = '/optics-oper/optics-ports/optics-port[name=Optics0/2/0/*]/optics-info'
    subscription_mode = "target_defined"
  [[inputs.gnmi.subscription]]
    name="Cisco-controller-otu"
    origin = "Cisco-IOS-XR-controller-otu-oper"
    path = '/otu/controllers/controller[controller-name=CoherentDSP0/2/0/*]/info'
    subscription_mode = "target_defined"

Logs from Telegraf

023-10-03T09:08:02Z D! [inputs.gnmi] Connection to gNMI device 10.44.250.93:12346 established
2023-10-03T09:08:02Z W! [inputs.gnmi] Got empty metric-name for response, usually indicating
configuration issues as the response cannot be related to any subscription.
Please open an issue on https://github.com/influxdata/telegraf including your
device model and the following response data:
timestamp:1696324083211000000 prefix:{origin:"openconfig-interfaces"} update:{path:{elem:{name:"interfaces"} elem:{name:"interface" key:{key:"name" value:"FourHundredGigE0/2/0/3"}} elem:{name:"state"}} val:{json_ietf_val:"{\"description\":\"REDACTED \"}"}}
This message is only printed once.

System info

Telegraf 1.28.1 / Cisco ASR 9922

Docker

No response

Steps to reproduce

  1. Regroup all subscription under 1 input
  2. Telegraf will start but will emit the initial error message and "Could not serialize metric: invalid name"

Expected behavior

Telegraf doesn't create any error and we expect to get the desired data from the openconfig-interfaces.

Actual behavior

Telegraf display an error at startup, and we don't get the desired data from the openconfig-interfaces. Moreover, we get lots of errors like this : 2023-10-03T09:08:09Z D! [outputs.file] Could not serialize metric: invalid name 2023-10-03T09:08:09Z D! [outputs.file] Could not serialize metric: invalid name 2023-10-03T09:08:09Z D! [outputs.file] Could not serialize metric: invalid name 2023-10-03T09:08:09Z D! [outputs.file] Could not serialize metric: invalid name 2023-10-03T09:08:09Z D! [outputs.file] Could not serialize metric: invalid name 2023-10-03T09:08:09Z D! [outputs.file] Could not serialize metric: invalid name 2023-10-03T09:08:09Z D! [outputs.file] Could not serialize metric: invalid name

Additional info

The workaround we use is define 2 inputs gnmi like this :

[[inputs.gnmi]]
  addresses = [${addresses}]
  username = "xxx"
  password = "xxx"
  redial = "60s"
  encoding = "json_ietf"
  name_override = "Cisco"
  [[inputs.gnmi.subscription]]
    name = "ifdesc"
    origin = "openconfig-interfaces"
    path = '/interfaces/interface[name=FourHundredGigE*]/state/description'
    subscription_mode = "sample"
    sample_interval = "60s"
  [[inputs.gnmi.subscription]]
    name = "temperature"
    origin = "openconfig-platform"
    path = '/components/component[name=FourHundredGigE*]/state/temperature/avg'
    subscription_mode = "target_defined"

[[inputs.gnmi]]
  addresses = [${addresses}]
  username = "xxx"
  password = "xxx"
  redial = "60s"
  encoding = "json_ietf"
  [[inputs.gnmi.subscription]]
    name = "Cisco-optics-lanes"
    origin = "Cisco-IOS-XR-controller-optics-oper"
    path = '/optics-oper/optics-ports/optics-port[name=Optics0/2/0/*]/optics-lanes/optics-lane'
    subscription_mode = "target_defined"
  [[inputs.gnmi.subscription]]
    name = "Cisco-optics-info"
    origin = "Cisco-IOS-XR-controller-optics-oper"
    path = '/optics-oper/optics-ports/optics-port[name=Optics0/2/0/*]/optics-info'
    subscription_mode = "target_defined"
  [[inputs.gnmi.subscription]]
    name="Cisco-controller-otu"
    origin = "Cisco-IOS-XR-controller-otu-oper"
    path = '/otu/controllers/controller[controller-name=CoherentDSP0/2/0/*]/info'
    subscription_mode = "target_defined"
jorisd commented 1 year ago 
./gnmic -a 10.44.250.93:12346  --log -u xxx -p xxx --insecure subscribe --path  "/interfaces/interface[name=FourHundredGigE*]/state/description"  --format flat
2023/10/03 16:46:40.329007 [gnmic] version=0.31.5, commit=bb3eb78, date=2023-07-24T20:10:58Z, gitURL=https://github.com/openconfig/gnmic, docs=https://gnmic.openconfig.net
2023/10/03 16:46:40.329041 [gnmic] using config file ""
2023/10/03 16:46:40.329210 [gnmic] starting output type file
2023/10/03 16:46:40.329334 [gnmic] queuing target "10.44.250.93:12346"
2023/10/03 16:46:40.329359 [gnmic] subscribing to target: "10.44.250.93:12346"
2023/10/03 16:46:40.329454 [gnmic] starting target "10.44.250.93:12346" listener
2023/10/03 16:46:40.329511 [file_output:default-stdout] initialized file output: {"Cfg":{"FileName":"","FileType":"stdout","Format":"flat","Multiline":true,"Indent":"  ","Separator":"\n","SplitEvents":false,"OverrideTimestamps":false,"AddTarget":"","TargetTemplate":"","EventProcessors":null,"MsgTemplate":"","ConcurrencyLimit":1000,"EnableMetrics":false,"Debug":false}}
2023/10/03 16:46:40.353296 [gnmic] target "10.44.250.93:12346" gNMI client created
2023/10/03 16:46:40.354161 [gnmic] sending gNMI SubscribeRequest: subscribe='subscribe:{subscription:{path:{elem:{name:"interfaces"}  elem:{name:"interface"  key:{key:"name"  value:"FourHundredGigE*"}}  elem:{name:"state"}  elem:{name:"description"}}}}', mode='STREAM', encoding='JSON', to 10.44.250.93:12346

openconfig:/interfaces/interface[name=FourHundredGigE0/2/0/7]/state/description: REDACTED

openconfig:/interfaces/interface[name=FourHundredGigE0/2/0/3]/state/description: REDACTED

^C
received signal 'interrupt'. terminating...
srebhan commented 1 year ago

@jorisd can you please test the binary in PR #14091 available once CI finished the tests! Let me know if this fixes the issue for you!

jorisd commented 1 year ago

Hi @srebhan Ack! Me and my team mate need more time to debug, but 1st feedback is I still see Warning at startup time, and Could not serialize metric: invalid name errors. However, it's for a different metric now. I'll get back to you soon next week !

srebhan commented 1 year ago

@jorisd please open a new issue if it's for a different metric/device. It would be also good to know the model and brand of the device if we need to enable some quirks to workaround stuff...