Open rgbkrk opened 9 years ago
vnstat
is a tool, but we likely need something in go to help us with this.
# vnstat -l -i bond0.101
Monitoring bond0.101... (press CTRL-C to stop)
rx: 12 kbit/s 29 p/s tx: 0 kbit/s 0 p/s^C
bond0.101 / traffic statistics
rx | tx
--------------------------------------+------------------
bytes 434 KiB | 64 KiB
--------------------------------------+------------------
max 56 kbit/s | 4 kbit/s
average 10.99 kbit/s | 1.62 kbit/s
min 4 kbit/s | 0 kbit/s
--------------------------------------+------------------
packets 9022 | 184
--------------------------------------+------------------
max 155 p/s | 3 p/s
average 28 p/s | 0 p/s
min 7 p/s | 0 p/s
--------------------------------------+------------------
time 5.27 minutes
vnStat is a console-based network traffic monitor. It keeps a log of hourly, daily and monthly network traffic for the selected interface(s). However, it isn't a packet sniffer. The traffic information is analyzed from the proc(5) and sys filesystems depending on availability. That way vnStat can be used even without root permissions on most systems.
Well neat, should have assumed that.
Ah, and ifconfig -a
can gives you a summary over an interface that I normally ignore:
veth46b4cbb Link encap:Ethernet HWaddr 26:24:ed:7e:fa:0b
inet6 addr: fe80::2424:edff:fe7e:fa0b/64 Scope:Link
UP BROADCAST RUNNING MTU:1500 Metric:1
RX packets:8 errors:0 dropped:0 overruns:0 frame:0
TX packets:16 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:648 (648.0 B) TX bytes:1296 (1.2 KB)
veth59e82d1 Link encap:Ethernet HWaddr ca:61:f6:8b:9b:74
inet6 addr: fe80::c861:f6ff:fe8b:9b74/64 Scope:Link
UP BROADCAST RUNNING MTU:1500 Metric:1
RX packets:1324831 errors:0 dropped:0 overruns:0 frame:0
TX packets:1258973 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:1061029350 (1.0 GB) TX bytes:382474515 (382.4 MB)
veth76ed421 Link encap:Ethernet HWaddr fe:90:10:3e:e1:a9
inet6 addr: fe80::fc90:10ff:fe3e:e1a9/64 Scope:Link
UP BROADCAST RUNNING MTU:1500 Metric:1
RX packets:8 errors:0 dropped:0 overruns:0 frame:0
TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:648 (648.0 B) TX bytes:648 (648.0 B)
RX bytes:648 (648.0 B) TX bytes:1296 (1.2 KB)
The container still has to be up though.
Now I wish I knew about how the Linux kernel reports this information so we could collect the bandwidth numbers easily. Won't solve the DC problem though...
Welp, we could use a libpcap wrapper for Go and sum the bytes on the interface. Then if we wanted to expose controls to demarcate IP blocks to ignore we could. Not terrible.
Actually, even better is gopacket. That's actually what packetbeat uses.
Yeah, that wasn't too bad to get started.
:+1: Awesome! How should we integrate this?
We could determine the interface and do the IP sniffing on container creation, all from the API node.
There's a big issue with trying to monitor the interface from the API node if we're using Docker swarm though. Kind of making me think we'd need to run our own job on each node.
I also have no idea how many CPU cycles or RAM we'd be wasting on monitoring. For now I could integrate it and we roll with it from there, maybe defaulting the monitoring to off.
We should meter bandwidth in and out.
Bandwidth in should be the sum of:
The last two can be monitored via their network interface. I'm not sure what tools are available other than summing on a
tcpdump
of the interface Docker gives them. The other hard bit is determining what kind of traffic it is. Do we want to regard traffic internal to a data center differently than traffic to the whole web?