Error in plugin: aborted gNMI subscription: rpc error: code = Unavailable desc = error reading from server: EOF

[mohsin106]

I’m using the gNMI plugin to connect to Juniper QFX devices. The device we are testing with is running version 20 of Junos code. I am able to run the gnmic command line client to connect to the QFX device and streaming some metrics back like this:

gnmic -a router1.mgt.net -u userName -p passWord --tls-ca /home/router_ca.pem --port 50051 subscribe --path /interfaces/interface

But when I try to use Telegraf version 1.22.4 I get the following error:

2022-05-25T19:22:34Z D! [inputs.gnmi] Connection to gNMI device router1.mgt.net:50051 established
2022-05-25T19:22:34Z D! [inputs.gnmi] Connection to gNMI device router1.mgt.net:50051 closed
2022-05-25T19:22:34Z E! [inputs.gnmi] Error in plugin: aborted gNMI subscription: rpc error: code = Unavailable desc = error reading from server: EOF
2022-05-25T19:22:34Z D! [inputs.gnmi] Connection to gNMI device router2.mgt.net:50051 established
2022-05-25T19:22:34Z D! [inputs.gnmi] Connection to gNMI device router2.mgt.net:50051 closed
2022-05-25T19:22:34Z E! [inputs.gnmi] Error in plugin: aborted gNMI subscription: rpc error: code = Unavailable desc = error reading from server: EOF
2022-05-25T19:22:35Z D! [inputs.gnmi] Connection to gNMI device router3.mgt.net:50051 established
2022-05-25T19:22:35Z D! [inputs.gnmi] Connection to gNMI device router3.mgt.net:50051 closed
2022-05-25T19:22:35Z E! [inputs.gnmi] Error in plugin: aborted gNMI subscription: rpc error: code = Unavailable desc = error reading from server: EOF
2022-05-25T19:22:36Z D! [outputs.influxdb] Buffer fullness: 0 / 15000 metrics
2022-05-25T19:22:37Z D! [outputs.kafka] Buffer fullness: 0 / 15000 metrics
2022-05-25T19:22:38Z D! [outputs.kafka] Buffer fullness: 0 / 15000 metrics
2022-05-25T19:22:44Z D! [inputs.gnmi] Connection to gNMI device router1.mgt.net:50051 established
2022-05-25T19:22:44Z D! [inputs.gnmi] Connection to gNMI device router1.mgt.net:50051 closed

My Telegraf Config:

[global_tags]
[agent]
  round_interval = true
  metric_batch_size = 1000
  metric_buffer_limit = 15000
  collection_jitter = "5s"
  flush_interval = "1m"
  flush_jitter = "5s"
  debug = false
  quiet = false
  hostname = "$containerName-telegraf-agent"
  omit_hostname = false

[[inputs.gnmi]]
  addresses = ["router1.mgt.net:50051", "router2.mgt.net:50051", "router3.mgt.net:50051"]
  username = "$routerUser"
  password = "$routerPass"
  redial = "10s"
  tls_ca = "/etc/telegraf/router_ca.pem"
  insecure_skip_verify = true

  [[inputs.gnmi.subscription]]
    name = "interfaces"
    origin = "openconfig-interfaces"
    path = "/interfaces/interface"
    subscription_mode = "sample"
    sample_interval = "10s"
    heartbeat_interval = "5s"

[[outputs.influxdb]]
  urls = ["http://10.63.60.32:8086"]
  database = "lab-qfx-gnmi"
  timeout = "45s"

Not sure if the problem is something on the device side or Telegraf side.

[powersj]

It looks like you have 3 different servers, are the other two operating as expected?

rpc error: code = Unavailable desc

This isn't exactly helpful from the server, but it does show that Telegraf is attempting to connect.

What is different between this server and the other two?

[mohsin106]

None of the servers are working. Server 1 and Server 3 are of the same type (RPA, which is a type of QFX device). Server 2 is a DPA (which is another type of QFX device). They all give the same Unavailable desc = error reading from server: EOF error message. Should I be looking into maybe gRPC version compatibility between what is running in Telegraf and what is running on the network device?

[powersj]

Should I be looking into maybe gRPC version compatibility between what is running in Telegraf and what is running on the network device?

Yeah that sounds like a good next step. Also can you confirm the versions used satisfy the README's requirements:

It has been optimized to support gNMI telemetry as produced by Cisco IOS XR (64-bit) version 6.5.1, Cisco NX-OS 9.3 and Cisco IOS XE 16.12 and later.

[telegraf-tiger[bot]]

Hello! I am closing this issue due to inactivity. I hope you were able to resolve your problem, if not please try posting this question in our Community Slack or Community Page. Thank you!

[donbharath21]

Hi, Here iam using the gnmi plugin to push the telemetry subscribed counters and admin-state in the influxdb. After configuring the telegraf.conf file in the influxdb i can my pulgins are enabled but i couldn't see any proper option.

For example: I am enabling the [inputs.gnmi.subscription] & [inputs.gnmi.tag_subscription] and in the influxdb query trying for the admin-state and oper-state but no option are there.

And when i try to start the telegraf it shows an error: $ telegraf --config http://172.30.25.93:8086/api/v2/telegrafs/0bebcfdac8d3d000 2023-10-20T01:21:51Z I! Loading config: http://172.30.25.93:8086/api/v2/telegrafs/0bebcfdac8d3d000 2023-10-20T01:21:51Z I! Starting Telegraf 1.28.2 brought to you by InfluxData the makers of InfluxDB 2023-10-20T01:21:51Z I! Available plugins: 240 inputs, 9 aggregators, 29 processors, 24 parsers, 59 outputs, 5 secret-stores 2023-10-20T01:21:51Z I! Loaded inputs: gnmi system 2023-10-20T01:21:51Z I! Loaded aggregators: 2023-10-20T01:21:51Z I! Loaded processors: 2023-10-20T01:21:51Z I! Loaded secretstores: 2023-10-20T01:21:51Z I! Loaded outputs: influxdb_v2 2023-10-20T01:21:51Z I! Tags enabled: host=sfsintel 2023-10-20T01:21:51Z I! [agent] Config: Interval:10s, Quiet:false, Hostname:"sfsintel", Flush Interval:10s 2023-10-20T01:21:51Z E! [inputs.gnmi] Error in plugin: failed to setup subscription: rpc error: code = Unavailable desc = connection error: desc = "error reading server preface: EOF"