search

python-diamond / Diamond

Diamond is a python daemon that collects system metrics and publishes them to Graphite (and others). It is capable of collecting cpu, memory, network, i/o, load and disk metrics. Additionally, it features an API for implementing custom collectors for gathering metrics from almost any source.
http://diamond.readthedocs.org/
MIT License
1.74k stars 601 forks source link

NetstatCollector is to slow for heavy loaded servers #660

Closed HumanUser closed 4 years ago

HumanUser commented 7 years ago

Hello. Current realization of NetstatCollector works to slow on high loaded server.

It's utilizing 100% of one core for about a minute:

$ wc -l /proc/net/tcp
115170 /proc/net/tcp
>>> from netstat import NetstatCollector
>>> nc = NetstatCollector
>>> nc = NetstatCollector()
>>> nc.collect()
>>> from datetime import datetime
>>> start=datetime.now();nc.collect();end=datetime.now();print((end-start).total_seconds())
58.351364

Mb it is better to make some ss-dependent version of this collector? (as it already linux-only based)

$ time ss|awk '{print $2}'|sort|uniq -c
    287 CLOSE-WAIT
    321 CLOSING
  96433 ESTAB
   9199 FIN-WAIT-1
   5292 FIN-WAIT-2
   1649 LAST-ACK
      1 State
      1 SYN-SENT

real    0m2.360s
user    0m2.072s
sys 0m0.432s