search

prometheus / node_exporter

Exporter for machine metrics
https://prometheus.io/
Apache License 2.0
10.69k stars 2.31k forks source link

IP fragmentation metrics stats missing #3012

Closed tekert closed 2 months ago

tekert commented 2 months ago

Host operating system: output of uname -a

Any linux machine.

node_exporter version: output of node_exporter --version

node_exporter, version 1.8.0 (branch: HEAD, revision: cadb1d1190ad95c66b951758f01ff4c94e55e6ce)
  build user:       root@64643165898e
  build date:       20240424-13:17:08
  go version:       go1.22.2
  platform:         linux/mipsle
  tags:             unknown

I noticed that the netstat collector says it exports netstat -s but they are mostly TCP and UDP stats, no IP fragmentation and reassembly statistics

For example in SNMP we have:


From IP-MIB DEFINITIONS

IP IN Reassembly stats:     Short Description
ipSystemStatsReasmReqds     The number of IP fragments received that needed to be reassembled at this interface
ipSystemStatsReasmOKs       The number of IP datagrams successfully reassembled
ipSystemStatsReasmFails     The number of failures detected by the IP re-assembly algorithm (for whatever reason: timed out, errors, etc.)

IP OUT Fragmentation Stats:
ipSystemStatsOutFragReqds   The number of IP datagrams that would require fragmentation in order to be transmitted
ipSystemStatsOutFragOKs     The number of IP datagrams that have been successfully fragmented
ipSystemStatsOutFragFails   The number of IP datagrams that have been discarded because they needed to be fragmented but could not be
ipSystemStatsOutFragCreates The number of output datagram fragments that have been generated as a result of IP fragmentation

For IPv4 and IPv6

There is more information inside the MIB with explanations.

With netstat -s we have: 

NETSTAT -S          equivalent? SNMP
reassemblies required       ipSystemStatsReasmReqds
packets reassembled ok      ipSystemStatsReasmOKs
fragments received ok       ipSystemStatsOutFragOKs
fragments failed        ipSystemStatsOutFragFails
fragments created       ipSystemStatsOutFragCreates

But no ipSystemStatsOutFragReqds or ipSystemStatsReasmFails

It's there any way to get these metrics from node_exporter? they are all very valuable for us at least.

dswarbrick commented 2 months ago

Adjust your --collector.netstat.fields flag to include the fragmentation stats (cf. /proc/net/snmp{,6}). You should be able to get something like:

# HELP node_netstat_Ip_FragCreates Statistic IpFragCreates.
# TYPE node_netstat_Ip_FragCreates untyped
node_netstat_Ip_FragCreates 2
# HELP node_netstat_Ip_FragFails Statistic IpFragFails.
# TYPE node_netstat_Ip_FragFails untyped
node_netstat_Ip_FragFails 0
# HELP node_netstat_Ip_FragOKs Statistic IpFragOKs.
# TYPE node_netstat_Ip_FragOKs untyped
node_netstat_Ip_FragOKs 1
# HELP node_netstat_Ip_ReasmFails Statistic IpReasmFails.
# TYPE node_netstat_Ip_ReasmFails untyped
node_netstat_Ip_ReasmFails 0
# HELP node_netstat_Ip_ReasmOKs Statistic IpReasmOKs.
# TYPE node_netstat_Ip_ReasmOKs untyped
node_netstat_Ip_ReasmOKs 0
# HELP node_netstat_Ip_ReasmReqds Statistic IpReasmReqds.
# TYPE node_netstat_Ip_ReasmReqds untyped
node_netstat_Ip_ReasmReqds 0
# HELP node_netstat_Ip_ReasmTimeout Statistic IpReasmTimeout.
# TYPE node_netstat_Ip_ReasmTimeout untyped
node_netstat_Ip_ReasmTimeout 0
tekert commented 2 months ago

oh, noticed it needs "Ip(6|)_" format to scrape those stats, does it srapes them from /proc/net/snmp ?

Adding ...|Ip(6|)_(Reasm.*|Frag.*)|... did the trick.

dswarbrick commented 2 months ago

The netstat collector reads from /proc/net/netstat, /proc/net/snmp and /proc/net/snmp6. See source: https://github.com/prometheus/node_exporter/blob/master/collector/netstat_linux.go

You can simplify Ip(6|)_(Reasm.*|Frag.*) to Ip6?_(Reasm|Frag).*.

discordianfish commented 2 months ago

I think this can be closed then