search

influxdata / telegraf

Agent for collecting, processing, aggregating, and writing metrics, logs, and other arbitrary data.
https://influxdata.com/telegraf
MIT License
14.62k stars 5.58k forks source link

Cisco_telemetry_mdt input plugin incorrect parsing of child policy #15387

Open GeorgeMaged opened 5 months ago

GeorgeMaged commented 5 months ago

Relevant telegraf.conf

###############################################################################
#                            SERVICE INPUT PLUGINS                            #
###############################################################################

[[inputs.cisco_telemetry_mdt]]
transport = "tcp"
service_address = ":5432"
max_msg_size=20000000
embedded_tags = [
"Cisco-IOS-XR-qos-ma-oper:qos/interface-table/interface/input/service-policy-names/service-policy-instance/statistics/class-stats/class-name",
"Cisco-IOS-XR-qos-ma-oper:qos/interface-table/interface/output/service-policy-names/service-policy-instance/statistics/class-stats/class-name",
"Cisco-IOS-XR-qos-ma-oper:qos/interface-table/interface/member-interfaces/member-interface/input/service-policy-names/service-policy-instance/statistics/class-stats/class-name",
"Cisco-IOS-XR-qos-ma-oper:qos/interface-table/interface/member-interfaces/member-interface/output/service-policy-names/service-policy-instance/statistics/class-stats/class-name"
]

[inputs.cisco_telemetry_mdt.aliases]
   cpu = "Cisco-IOS-XR-wdsysmon-fd-oper:system-monitoring/cpu-utilization"
   memory-summary = "Cisco-IOS-XR-nto-misc-oper:memory-summary/nodes/node/summary"
   memory-detail = "Cisco-IOS-XR-nto-misc-shmem-oper:memory-summary/nodes/node/detail"
   interface_QoS_In = "Cisco-IOS-XR-qos-ma-oper:qos/interface-table/interface/input/service-policy-names/service-policy-instance/statistics"
   interface_QoS_Out = "Cisco-IOS-XR-qos-ma-oper:qos/interface-table/interface/output/service-policy-names/service-policy-instance/statistics"
   if_stats = "Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/latest/generic-counters"

Logs from Telegraf

.

System info

Telegraf 1.20.4 , Debian GNU/Linux 10 (buster) Kernel: Linux 4.19.0-6-amd64

Docker

No response

Steps to reproduce

1.Cisco equipment details Cisco IOS XR Software, Version 6.5.3 Version : 6.5.3

2.Configuration in Router :

Router(config)#telemetry model-driven Router(config-model-driven)#destination-group GROUP1 Router(config-model-driven-dest)#address family ipv4 10.53.225.254 port 5432
Router(config-model-driven-dest-addr)#encoding self-describing-gpb
Router(config-model-driven-dest-addr)#protocol tcp
Router(config-model-driven-dest-addr)#commit

Sensor Groups: Sensor Group Id:S-Group1 Sensor Path: Cisco-IOS-XR-wdsysmon-fd-oper:system-monitoring/cpu-utilization Sensor Path State: Resolved

Sensor Group Id:S-Group2 Sensor Path: Cisco-IOS-XR-nto-misc-shmem-oper:memory-summary/nodes/node/detail Sensor Path State: Resolved

Sensor Group Id:S-Group3 Sensor Path: Cisco-IOS-XR-qos-ma-oper:qos/interface-table/interface/input/service-policy-names/service-policy-instance/statistics Sensor Path State: Resolved

Sensor Group Id:S-Group4 Sensor Path: Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/latest/generic-counters Sensor Path State: Resolved

Sensor Group Id:S-Group5 Sensor Path: Cisco-IOS-XR-qos-ma-oper:qos/interface-table/interface/output/service-policy-names/service-policy-instance/statistics Sensor Path State: Resolved Sensor Path: Cisco-IOS-XR-qos-ma-oper:qos/interface-table/interface/output/service-policy-names/service-policy-instance/statistics Sensor Path State: Resolved

Subscribe to these sensor groups.

  1. Send these data into influxDB.

Expected behavior

The expected behaviour is to be able to parse the different classes in the child policy. Expected to get all class names related to child class name but we receive only "class default"

Actual behavior

convert only class name for parent policy not child and hence get only all class names for parent not child policy .

Additional info

No response

srebhan commented 5 months ago

@GeorgeMaged did you try with a more recent version of Telegraf?

GeorgeMaged commented 5 months ago

@GeorgeMaged did you try with a more recent version of Telegraf?

Actually, I never thought that the issue could be from the version of telegraf. Can a newer version resolve this issue of sensor path parsing till child policy and the internal QoS ?

powersj commented 5 months ago

Can a newer version resolve this issue of sensor path parsing till child policy and the internal QoS ?

We are asking you to try a newer version and let us know. We do not have access to the Cisco devices directly.

telegraf-tiger[bot] commented 4 months ago

Hello! I am closing this issue due to inactivity. I hope you were able to resolve your problem, if not please try posting this question in our Community Slack or Community Forums or provide additional details in this issue and reqeust that it be re-opened. Thank you!

GeorgeMaged commented 2 weeks ago

Hello there, I tried using Telegraf v1.32 and tested it and still it parses correctly the sensor path of qos but only (class-default) is retreived from the child policy classes while the other child classes are not retreived. If the parent policy only, it retrieves correctly alll the qos data of the different classes of parent policy , the problem exists in the child ones

srebhan commented 1 week ago

@GeorgeMaged could you please capture one of the messages for me to reproduce the issue?

GeorgeMaged commented 1 week ago

@srebhan That was the output from telegraf:

"fields":{"class_stats/cac_state":"unknown","class_stats/child_policy/content/class_stats/cac_state":"unknown","class_stats/child_policy/content/class_stats/class_name":"class-default","class_stats/child_policy/content/class_stats/counter_validity_bitmask":4783898624,"class_stats/child_policy/content/class_stats/general_stats/match_data_rate":0,"class_stats/child_policy/content/class_stats/general_stats/pre_policy_matched_bytes":458593,"class_stats/child_policy/content/class_stats/general_stats/pre_policy_matched_packets":1231,"class_stats/child_policy/content/class_stats/general_stats/total_drop_bytes":0,"class_stats/child_policy/content/class_stats/general_stats/total_drop_packets":0,"class_stats/child_policy/content/class_stats/general_stats/total_drop_rate":0,"class_stats/child_policy/content/class_stats/general_stats/total_transmit_rate":0,"class_stats/child_policy/content/class_stats/general_stats/transmit_bytes":458593,"class_stats/child_policy/content/class_stats/general_stats/transmit_packets":1231,"class_stats/child_policy/content/class_stats/police_stats_array/conform_bytes":56654502934,"class_stats/child_policy/content/class_stats/police_stats_array/conform_packets":765601391,"class_stats/child_policy/content/class_stats/police_stats_array/conform_rate":10,"class_stats/child_policy/content/class_stats/police_stats_array/drop_bytes":0,"class_stats/child_policy/content/class_stats/police_stats_array/drop_packets":0,"class_stats/child_policy/content/class_stats/police_stats_array/exceed_bytes":0,"class_stats/child_policy/content/class_stats/police_stats_array/exceed_packets":0,"class_stats/child_policy/content/class_stats/police_stats_array/exceed_rate":0,"class_stats/child_policy/content/class_stats/police_stats_array/parent_drop_bytes":0,"class_stats/child_policy/content/class_stats/police_stats_array/parent_drop_packets":0,"class_stats/child_policy/content/class_stats/police_stats_array/violate_bytes":0,"class_stats/child_policy/content/class_stats/police_stats_array/violate_packets":0,"class_stats/child_policy/content/class_stats/police_stats_array/violate_rate":0,"class_stats/child_policy/content/class_stats/queue_stats_array/conform_bytes":458593,"class_stats/child_policy/content/class_stats/queue_stats_array/conform_packets":1231,"class_stats/child_policy/content/class_stats/queue_stats_array/conform_rate":0,"class_stats/child_policy/content/class_stats/queue_stats_array/exceed_bytes":0,"class_stats/child_policy/content/class_stats/queue_stats_array/exceed_packets":0,"class_stats/child_policy/content/class_stats/queue_stats_array/exceed_rate":0,"class_stats/child_policy/content/class_stats/queue_stats_array/forced_wred_stats_display":"true","class_stats/child_policy/content/class_stats/queue_stats_array/max_threshold_bytes":0,"class_stats/child_policy/content/class_stats/queue_stats_array/max_threshold_packets":0,"class_stats/child_policy/content/class_stats/queue_stats_array/queue_average_length/unit":"policy-param-unit-packets","class_stats/child_policy/content/class_stats/queue_stats_array/queue_average_length/value":0,"class_stats/child_policy/content/class_stats/queue_stats_array/queue_drop_threshold":0,"class_stats/child_policy/content/class_stats/queue_stats_array/queue_id":131158,"class_stats/child_policy/content/class_stats/queue_stats_array/queue_instance_length/unit":"policy-param-unit-packets","class_stats/child_policy/content/class_stats/queue_stats_array/queue_instance_length/value":0,"class_stats/child_policy/content/class_stats/queue_stats_array/random_drop_bytes":0,"class_stats/child_policy/content/class_stats/queue_stats_array/random_drop_packets":0,"class_stats/child_policy/content/class_stats/queue_stats_array/tail_drop_bytes":0,"class_stats/child_policy/content/class_stats/queue_stats_array/tail_drop_packets":0,"class_stats/child_policy/content/class_stats/shared_queue_id":4294967295,"class_stats/child_policy/content/class_stats/wred_stats_array/max_threshold_bytes":0,"class_stats/child_policy/content/class_stats/wred_stats_array/max_threshold_packets":0,"class_stats/child_policy/content/class_stats/wred_stats_array/profile_title":"Default WRED Curve","class_stats/child_policy/content/class_stats/wred_stats_array/random_drop_bytes":0,"class_stats/child_policy/content/class_stats/wred_stats_array/random_drop_packets":0,"class_stats/child_policy/content/class_stats/wred_stats_array/red_ecn_marked_bytes":0,"class_stats/child_policy/content/class_stats/wred_stats_array/red_ecn_marked_packets":0,"class_stats/child_policy/content/class_stats/wred_stats_array/red_label/value":0,"class_stats/child_policy/content/class_stats/wred_stats_array/red_label/wred_type":"red-with-default-min-max","class_stats/child_policy/content/class_stats/wred_stats_array/red_transmit_bytes":0,"class_stats/child_policy/content/class_stats/wred_stats_array/red_transmit_packets":0,"class_stats/child_policy/content/policy_name":"WAN-LINKS-QOS-OUT-NCS-PQ","class_stats/child_policy/content/policy_timestamp":0,"class_stats/child_policy/content/satid":0,"class_stats/child_policy/content/state":"active","class_stats/child_policy/keys/interface_name":"Bundle-Ether2001.2113","class_stats/child_policy/keys/service_policy_name":"NCS-LAN-WAN-QOS-SHAPE-1G","class_stats/counter_validity_bitmask":2097152,"class_stats/general_stats/pre_policy_matched_bytes":56661176795,"class_stats/general_stats/pre_policy_matched_packets":765644736,"class_stats/general_stats/total_drop_bytes":0,"class_stats/general_stats/total_drop_packets":0,"class_stats/general_stats/transmit_bytes":56661174279,"class_stats/general_stats/transmit_packets":765644702,"match_data_rate":10,"policy_name":"NCS-LAN-WAN-QOS-SHAPE-1G","policy_timestamp":0,"satid":0,"state":"active","total_drop_rate":0,"total_transmit_rate":10},"name":"interface_QoS_Out","tags":{"class_stats/class_name":"class-default","host":"timor","interface_name":"Bundle-Ether2001.2113","path":"Cisco-IOS-XR-qos-ma-oper:qos/interface-table/interface/output/service-policy-names/service-policy-instance/statistics","service_policy_name":"NCS-LAN-WAN-QOS-SHAPE-1G","source":"SWH01-PE02","subscription":"SUB6"},"timestamp":1729060750}

Only class-default from the child classes is retrieved, while the child policy has many classes .

srebhan commented 1 week ago

Sorry for not being clear, I meant the input side. E.g. a tcpdump of one of the packets going into telegraf... This way I can (mock) replay your data and see what is going on in the plugin and why things are missing...

GeorgeMaged commented 1 week ago

Sure I will provide it, but till then I checked it already and found that data was sent to the server before going into telegraf but i will provide sample of the dump file to check it also.

GeorgeMaged commented 1 week ago

@srebhan This is one of the tcpdump results for a packet but from Wireshark the output is something like this:

Ýù conform-rate8z exceed-rate8z¥ class-statsz counter-validity-bitmask@éz _class-name class-default_zshared-queue-id8ÿÿÿÿz cac-stateunknownzî general-statsztransmit-packets@çztransmit-bytes@Àztotal-drop-packets@ztotal-drop-bytes@ztotal-drop-rate8zmatch-data-rate8ztotal-transmit-rate8zpre-policy-matched-packets@çzpre-policy-matched-bytes@Àzñqueue-stats-arrayzqueue-id8æ ztail-drop-packets@ztail-drop-bytes@zEqueue-instance-lengthz value8z!unitpolicy-param-unit-packetszDqueue-average-lengthz value8z!unitpolicy-param-unit-packetszqueue-drop-threshold8z!forced-wred-stats-displaytruezrandom-drop-packets@zrandom-drop-bytes@zmax-threshold-packets@zmax-threshold-bytes@zconform-packets@çz conform-bytes@Àzexceed-packets@z exceed-bytes@z conform-rate8z exceed-rate8zÂwred-stats-arrayz# profile-titleDefault WRED Curvez= red-labelz% wred-typered-with-default-min-maxz value8zred-transmit-packets@zred-transmit-bytes@zrandom-drop-packets@zrandom-drop-bytes@zmax-threshold-packets@zmax-threshold-bytes@zred-ecn-marked-packets@zred-ecn-marked-bytes@z satid8zpolicy-timestamp@z satid8zpolicy-timestamp@ZDÖ¦½ü¨2z]keysz'interface-nameBundle-Ether2004.2410z,service-policy-nameBundle-Ether2004.2410z®Ccontentz' policy-nameNCS-LAN-WAN-QOS-SHAPE-1GzstateactivezÇB class-statszcounter-validity-bitmask@z class-name class-defaultz cac-stateunknownzú general-statsztransmit-packets@̺Ûztransmit-bytes@Õª³¸ztotal-drop-packets@ztotal-drop-bytes@Jztotal-drop-rate8zmatch-data-rate8 ztotal-transmit-rate8 z"pre-policy-matched-packets@±ºÛz!pre-policy-matched-bytes@ÈŪ³¸zæ? child-policyz]keysz'interface-nameBundle-Ether2004.2410z,service-policy-nameBundle-Ether2004.2410zö>contentz' policy-nameWAN-LINKS-QOS-OUT-NCS-PQzstateactivez class-statsz counter-validity-bitmask@ Áz _class-nameVOIP-SIG_zshared-queue-id8ÿÿÿÿz cac-state*unknownzö general-statsztransmit-packets@Û¦…ztransmit-bytes@¥¿Çõztotal-drop-packets@ztotal-drop-bytes@ztotal-drop-rate8zmatch-data-rate8ztotal-transmit-rate8z!pre-policy-matched-packets@Û¦…z pre-policy-matched-bytes@¥¿Çõzõqueue-stats-arrayzqueue-id8Ø ztail-drop-packets@ztail-drop-bytes@zEqueue-instance-lengthz

vkrasnici commented 1 week ago

@GeorgeMaged - can you please attach the output taken from the router: run mdt_exec -s Cisco-IOS-XR-qos-ma-oper:qos/interface-table/interface/output/service-policy-names/service-policy-instance/statistics or run mdt_exec -s Cisco-IOS-XR-qos-ma-oper:qos/interface-table/interface/output/service-policy-names/service-policy-instance/statistics if your checking for output policy. Also, what is your output format and plugin?

GeorgeMaged commented 1 day ago

@vkrasnici Here's an output from the router:

child-policy":{"policy-name":"WAN-LINKS-QOS-OUT-NCS-PQ","state":"active","class-stats":[{"counter-validity-bitmask":"4700016640","class-name":"VOIP-SIG","shared-queue-id":4294967295,"cac-state":"unknown","general-stats "child-policy":{"keys":[{"interface-name":"Bundle-Ether2001.2111"},{"service-policy-name":"NCS-LAN-WAN-QOS-SHAPE-1G"}],"content":{"policy-name":"WAN-LINKS-QOS-OUT-NCS-PQ","state":"active","class-stats":[{"counter-validity-bitmask":"4700016640","class-name":"VOIP-SIG","shared-queue-id":4294967295,"cac-state":"unknown","general-stats":{"transmit-packets":"221599","transmit-bytes":"32908988","total-drop-packets":"0","total-drop-bytes":"0","total-drop-rate":0,"match-data-rate":0,"total-transmit-rate":0,"pre-policy-matched-packets":"221599","pre-policy-matched-bytes":"32908988"},"queue-stats-array":[{"queue-id":131168,"tail-drop-packets":"0","tail-drop-bytes":"0","queue-instance-length":[{"value":0,"unit":"policy-param-unit-packets"}],"queue-average-length":[{"value":0,"unit":"policy-param-unit-packets"}]

I am using outputs.influxdb plugin.