search

david415 / HoneyBadger

Quantum Insert detector/recorder
GNU General Public License v3.0
305 stars 39 forks source link

super FAST concurrent design with AF_PACKET fanout! #64

Open david415 opened 9 years ago

david415 commented 9 years ago

OMG I shoulda done this months ago! I do not understand why there exists AF_PACKET and PF_RING... they are essentially equivalent but AF_PACKET is merged in the mainline kernel whereas PF_RING has a cult following of people who like to patch their kernels.

AF_PACKET has a flow collision hash ring for dispatching packets to the fanout group members. it is perfect for the honeybadger usage. honeybadger will spawn N sniffer goroutines which will become members of the same fanout group. each member then receives it's slice of the flow hash ring worth of connections! perfect. muhahaha!

david415 commented 9 years ago

beware of how IP packet fragmentation will affect things... : http://man7.org/linux/man-pages/man7/packet.7.html

IP fragmentation causes packets from the same flow to have different flow hashes.  The flag PACKET_FANOUT_FLAG_DEFRAG, if set, causes packets to be defragmented before fanout is applied, to preserve order even in this case.

Therefore it is my understanding that we must use the FanoutHashWithDefrag option defined here: https://godoc.org/github.com/google/gopacket/afpacket#FanoutType

david415 commented 8 years ago

Let's evaluate how much work this feature addition will be so that we can properly prioritize this feature addition. It may well be the easiest path for honeybadger to approach fairly high line speeds.

david415 commented 8 years ago

It was super easy to add this as an option here in my dev branch: https://github.com/david415/HoneyBadger/tree/64.afpacket_fanout_hash.0

The next step is to test this dev branch. I should probably ask people to help me test this.

We should perform two kinds of tests:

In my last commit that I changed the fanout type from afpacket.FanoutHash to afpacket.FanoutHashWithDefrag. That's because my brief cursory testing concluded that with afpacket.FanoutHash IPv6 was captured correctly and IPv4 packets were NOT properly captured but I didn't dig any further than that.

How fast is honeybadger if we run multiple procs in a fanout hash ring?

I'm sure there are speed improvements to be made, but before we start to optimize for speed we need to benchmark the current performance to understand which changes make improvements.