Open TurboTurtle opened 5 years ago
Need to investigate the available python modules for network sniffing so that we can avoid shelling-out to tshark
. I would like to avoid shelling-out in rigs as a general rule of thumb, though actions will likely make significant use of that approach.
Also need to see what is available in downstream distributions so that we don't run into packaging issues down the road.
Probably the most universal python module to achieve this would be Scapy. It supports bpf filters out of the box, and also gives access to all fields in all layers without the need to do extra string parsing of the output
Yeah, that's what I found as well, however the problem is that Scapy does not ship in RHEL currently. I opened an RFE a while back, but there has been no movement on it at all.
Well, it was a little bit heavyweight anyway :)
Getting rid of tcpdump will require at some point to rely on a wrapper for libpcap. There are quite a few out there, not sure if any of them is planned to be included in RHEL out of the box.
Worst case scenario, maybe one of these wrappers with the appropiate license can be just included as part of the rig package?
Reviving this issue :) I recently learned how to do this using just plain raw sockets. I'll try to provide a working prototype
@TurboTurtle as initial approach I'll try to provide command line options to define some basic matching options as IP address, port or protocol. Something like --ip 10.0.0.1
or --srcport 443/tcp
.
Allowing full user friendly bpf strings from the user and getting them translated into BPF JIT assembly without any 3rd party dependencies will require a lot of work and tunning.
See for example what a simple filter to match packets originating from a web service would translate to:
% sudo tcpdump -i any -n -d 'tcp and src port 80 or src port 443'
(000) ldh [14]
(001) jeq #0x86dd jt 2 jf 11
(002) ldb [22]
(003) jeq #0x6 jt 4 jf 6
(004) ldh [56]
(005) jeq #0x50 jt 27 jf 19
(006) jeq #0x2c jt 28 jf 7
(007) jeq #0x84 jt 9 jf 8
(008) jeq #0x11 jt 9 jf 28
(009) ldh [56]
(010) jeq #0x1bb jt 27 jf 28
(011) jeq #0x800 jt 12 jf 28
(012) ldb [25]
(013) jeq #0x6 jt 14 jf 20
(014) ldh [22]
(015) jset #0x1fff jt 28 jf 16
(016) ldxb 4*([16]&0xf)
(017) ldh [x + 16]
(018) jeq #0x50 jt 27 jf 19
(019) jeq #0x1bb jt 27 jf 28
(020) jeq #0x84 jt 22 jf 21
(021) jeq #0x11 jt 22 jf 28
(022) ldh [22]
(023) jset #0x1fff jt 28 jf 24
(024) ldxb 4*([16]&0xf)
(025) ldh [x + 16]
(026) jeq #0x1bb jt 27 jf 28
(027) ret #262144
(028) ret #0
The module would need to do the whole expression evaluation + translation to BPF asm in order to provide in-kernel filtering.
Another option would be to do just the expression evaluation and then doing the filtering in python after parsing at least layers 2, 3 and 4, which would not be even closely as efficient as compiling for the BPF JIT.
Do you have any commonly used filters, or are there any basic requirements that you have in mind? Would options --srcip
, --dstip
, --protocol
, --srcport
, --dstport
do the job for now?
I honestly haven't looked at this deeply in a while, mainly due to the lack of movement of getting an existing module to support it available. If we can do this with raw sockets ourselves, that's definitely interesting!
Basic requirements for an MVP here I think would be a simple "trigger when we get X type of packet on Y address/port (optionally from Z source)". So, yes, the options you've listed look to be perfect for that use case.
To be a little more specific in the type of use case I envisioned for this, "trigger when we get a (or X number of) RST packet(s) from srcip 10.11.12.13 on dstip 10.9.8.7"
"trigger when we get a (or X number of) RST packet(s) from srcip 10.11.12.13 on dstip 10.9.8.7"
Who does the counting? Is there any generic counter keeping track of the amount of times that a rig triggered, or should the rig accept the number via command line and keep track of the amount of matches itself?
I have a working prototype here https://github.com/juanmasg/rig/blob/network/rigging/rigs/network.py . Just IPs and ports for now. I'm working on trying to match the user provided TCP flags (e.g RST) with the packet. If you have some time in the next few days for doing a quick visual check of the rig code to make sure I'm not missing anything important that would be great. I believe that it implements everything, but not 100% sure. I can open a PR if you prefer.
I've taken a look over it, and that looks well designed! I haven't been able to actually test it yet, and it'll likely be next week before I have a chance to - but it's looking good so far. Nothing is jumping out at me, at least.
Feel free to post a PR, and I'll comment on it after I've been able to properly test it.
I've generated the PR. It includes the manpage updates as well. That will do it for now I think, besides bugfixes, etc. Everything seems to work as expected according to my tests. Anything, let me know
As discussed, these are my proposed network related names/rigs:
packet
: Performs network packet matching (the current network
rig would be renamed to this). Examples:
connection
: Match network connection attributes. Examples:
TIME_WAIT
exceeds the given threshold.netstat
: Network statistics. Examples:
netlink
: Match NETLINK
events. Examples:
tun0
appears or disappearsdown
on interface eth0
What do you think?
I like that list. I had some initial hesitance over packet
, but it makes more sense the more I think on it, and I continue to not be able to come up with a better alternative anyways :wink:
Placeholder to create a rig that monitors network activity. Initial idea is that users will provide a tshark packet filter string and once we get a single packet matching the filter, the rig will trigger.