search

buger / goreplay

GoReplay is an open-source tool for capturing and replaying live HTTP traffic into a test environment in order to continuously test your system with real data. It can be used to increase confidence in code deployments, configuration changes and infrastructure changes.
https://goreplay.org
Other
18.54k stars 13 forks source link

VXLAN support for GoReplay #1048

Closed stephanemartin closed 2 years ago

stephanemartin commented 2 years ago

Go replay can record with pcap and call middleware for each exchange. The middleware can analyze the traffic and make a report (debug, stats, and so on purpose).

This patch, provide the VXLAN support to capture the traffic. For example, if you are using AWS traffic mirroring, you can keep your middleware or forward to other goreplay.

The source ip source or port designed in '--input-raw' help to know the direction of the packet.

The 4789 UDP port is open and traffic is injected as with the pcap library.

./gor --input-raw :80 --input-raw-track-response --output-http-track-response --input-raw-engine vxlan --middleware my-favorite-middleware --output-null
./gor --input-raw :80 --input-raw-track-response --output-http-track-response --input-raw-engine vxlan -output-stdout

We have hard coded the port 4789 because: we haven't found satisfayeing method to add that in GoReplay arguments and it the default port of VXLAN.

CLAassistant commented 2 years ago

CLA assistant check
All committers have signed the CLA.

sonarcloud[bot] commented 2 years ago

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

buger commented 2 years ago

Hi!

That's very cool! I was looking to solve exactly the same issue recently.

I can help with proper way for configuring vlanx port, but do you think you can add a way for filtering traffic by specific VNI?

Thanks!

buger commented 2 years ago

Also alternative to this would be use classical traffic capture, like with TCP, then you even be able apply BPF filters, like tcp dump does: tcpdump -l -n -i <if> 'port 4789 and udp[8:2] = 0x0800 & 0x0800 and udp[11:4] = <vni> & 0x00FFFFFF', but I wonder about pros and cons of this approaches?

stephanemartin commented 2 years ago

Hi!

That's very cool! I was looking to solve exactly the same issue recently.

I can help with proper way for configuring vlanx port, but do you think you can add a way for filtering traffic by specific VNI?

Thanks!

Hello, Yes we can, that is the field of the packet. If I've a proper way to add options in goreplay args, I can filter that. And add the port customization too :)

Also alternative to this would be use classical traffic capture, like with TCP, then you even be able apply BPF filters, like tcp dump does: tcpdump -l -n -i <if> 'port 4789 and udp[8:2] = 0x0800 & 0x0800 and udp[11:4] = <vni> & 0x00FFFFFF', but I wonder about pros and cons of this approaches?

To be honnest, I've hesited with the approach to use the lib pcap to capture in GoReplay and decode vxlan (with gopacket). Without socket creating, the OS replies with "closed socket" udp packet, and if I use GoReplay to open it I've the information twice.

I haven't see a possibility to decapsulate the vxlan packet easily from pcap and capture.go... May be with more time.

Thanks for your feedback.

stephanemartin commented 2 years ago

So our aim is to pass configuration from settings.go to capture.go activateVxLanSocket() We can add to Listener structure inside input_raw.go We can add n th args to NewListener or pass by setter such as pcapOptions. new structure inside RAWInputConfig and filled in settings.go.

What do you think ?

buger commented 2 years ago

I applied your changes, and made needed modifications. Plus it also adds support for VLAN protocol. Pls review. Thanks!

https://github.com/buger/goreplay/pull/1051