search

emmericp / MoonGen

MoonGen is a fully scriptable high-speed packet generator built on DPDK and LuaJIT. It can saturate a 10 Gbit/s connection with 64 byte packets on a single CPU core while executing user-provided Lua scripts for each packet. Multi-core support allows for even higher rates. It also features precise and accurate timestamping and rate control.
MIT License
1.04k stars 235 forks source link

Fast packet capturing #180

Closed ztz1989 closed 7 years ago

ztz1989 commented 7 years ago

Hi, Paul. If I want to receive & capture packets at 10Gbit/s rate and write them to a pcap file using MoonGen, what kinds of MoonGen APIs should I use so as to write all the received packes to pcap file? Currently, I use queue:tryRecv(bufs, 100) to read 100 packets at a time from the queue to the bufs and use pcap:newWriter to write each received packet to a pcap file. But when I send packets at 10Gbit/s, I always capture less packets than I received. Thanks in advance!

emmericp commented 7 years ago

Did you have a look at the dump-pkts.lua example?

https://github.com/libmoon/libmoon/blob/master/examples/dump-pkts.lua

(probably needs a sym link in the moongen examples dir)

There are also a few tunables for the device configuration (device.config) to prevent packet loss:

I've managed to get ~11 Mpps with small packets to disk with a single core, scaling well with rss/multi-queue configurations.

ztz1989 commented 7 years ago

Hi, Paul. Thank you very much. I just tried this example with 10Gbit frame rate, and there is no packet loss.

ztz1989 commented 7 years ago

Hi, Paul. Just one last question out of curiosity: when I run the dump-pkts.lua example with single thread, I got output as follows:

[Capture, thread #1] RX: 0.00 (StdDev 0.00) Mpps, 0 (StdDev 0) Mbit/s (0 Mbit/s with framing), total 511 packets with 32704 bytes (incl. CRC) [Filter reject, thread #1] RX: 0.00 (StdDev 0.00) Mpps, 0 (StdDev 0) Mbit/s (0 Mbit/s with framing), total 0 packets with 0 bytes (incl. CRC) [INFO] Flushing buffers, this can take a while... [Device: id=2] RX: 0.00 (StdDev 0.00) Mpps, 0 (StdDev 0) Mbit/s (0 Mbit/s with framing), total 567 packets with 36288 bytes (incl. CRC)

The device counter got 567 packets while the packet counter reported only 511 packets. Is there a way to make the packet counter get the same number of packets as the device counter?

emmericp commented 7 years ago

if you are plugging this into live traffic: no, unfortunately not because DPDK needs to reset the NIC (and link) on startup, so the hardware counter will always somewhat higher

djin31 commented 6 years ago

I wished to run a script so that I can send packets at line rate and capture them in a pcap file at the other end of loopback. For that I wrote the following script adapted from https://github.com/libmoon/libmoon/blob/master/examples/dump-pkts.lua send-dump.zip The packet capturing task is done by function dumper. Commented along are codes for packet capturing portions from rx-to-pcap.lua as well as dump-pkt.lua. However all the three codes were not able to capture traffic properly.

The console output is like this for the dumper function I wrote: [Device: id=1] RX: 1.86 Mpps, 1253 Mbit/s (1551 Mbit/s with framing) [Device: id=0] TX: 1.87 Mpps, 1255 Mbit/s (1554 Mbit/s with framing) [Capture, thread #1] RX: 0.00 Mpps, 0 Mbit/s (0 Mbit/s with framing) [Device: id=1] RX: 2.16 Mpps, 1451 Mbit/s (1797 Mbit/s with framing) [Device: id=0] TX: 2.16 Mpps, 1451 Mbit/s (1797 Mbit/s with framing) [Capture, thread #1] RX: 0.00 Mpps, 0 Mbit/s (0 Mbit/s with framing) [Device: id=1] RX: 2.16 Mpps, 1451 Mbit/s (1797 Mbit/s with framing) [Device: id=0] TX: 2.16 Mpps, 1451 Mbit/s (1797 Mbit/s with framing) [Capture, thread #1] RX: 0.00 Mpps, 0 Mbit/s (0 Mbit/s with framing) ^Cdumper terminated after receiving 0 packets dumper terminated after receiving 0 packets [Capture, thread #2] RX: nan (StdDev 0.00) Mpps, nan (StdDev 0) Mbit/s (nan Mbit/s with framing), total 0 packets with 0 bytes (incl. CRC) [Capture, thread #1] RX: 0.00 (StdDev 0.00) Mpps, 0 (StdDev 0) Mbit/s (0 Mbit/s with framing), total 0 packets with 0 bytes (incl. CRC) [Device: id=1] RX: 2.16 (StdDev 0.00) Mpps, 1451 (StdDev 0) Mbit/s (1797 Mbit/s with framing), total 7820886 packets with 656954514 bytes (incl. CRC) [Device: id=0] TX: 2.16 (StdDev 0.00) Mpps, 1451 (StdDev 0) Mbit/s (1797 Mbit/s with framing), total 7820883 packets with 656954172 bytes (incl. CRC)

Can you please redirect to a certain example that shows how to capture packets in a pcap file while sending them from the other port?

emmericp commented 6 years ago

Tthere's nothing special about sending and dumping at the same time. Just copy and paste the threads from a send and a dump example together.

The script you attached makes little sense, you try to use queues that obviously don't receive any packets, you use a packet counter without counting packets and you have double frees of bufs all over the place.

djin31 commented 6 years ago

Sorry for the inconvenience, I was not aware of the fact that queues don't receive any packets. The confusion arose mainly due to the fact that the script in the examples of libmoon used queues for dumping into pcap file (or atleast I felt so).

emmericp commented 6 years ago

Our script configures RSS, yours doesn't. But the main problem are not counting packets and double frees.

djin31 commented 6 years ago

Thanks a lot for your suggestions. I got the script working . However, there was a drop rate of around 6-7% with a single thread with a rate of 10Mpps. I tried increasing the number of threads and increasing rxDescs, but it did not help as only one thread appears to be getting RX traffic while the other one keeps showing zero traffic. Can you please have a look and guide me to what I am doing wrong?

send-dump-pcap.zip

emmericp commented 6 years ago

Pleasse try to familiarize yourself with basic concepts of NICs and DPDK first. You are looking for the RSS feature. Or just copy and paste my existing code which properly configures the queues.

fabiolucianoverdi commented 1 year ago

Hello, I have a different setup: I want to capture the packets sent at the tx. The problem posted by @djin31 is using a loopback cable. In my case, I just want to capture the tx traffic leaving my device. Is it possible? We have tried to adapt a few code from @djin31 but no success so far. Any hint is more than welcome.

emmericp commented 1 year ago

you should be able to copy code from https://github.com/libmoon/libmoon/blob/master/examples/dump-pkts.lua and put the packets you are sending in both the tx queue and the pcap writer. the only important diff to the dumper example is to not free the packets explicitly, tx will take care of this

guimvmatos commented 1 year ago

Hi @emmericp ,

I'm working with Fabio (who sent the message above).

We want to capture the packet being sent as close to the time of sending as possible. We try to do like dump-pkts. However, I realized that he is using the tryRecv function in Rx. In my case, I would have to use the same function in TX, but when I do this change, I get an error. It seems that this function is only functional in the RX. How can I read the packets sent and save them in pcap. Here's my code (pay attention to the dumpSlave function).

fabiolucianoverdi commented 1 year ago

Hi @emmericp, just adding a print screen of the error that is happening regarding the problem pointed out by @guimvmatos. errorTx

fabiolucianoverdi commented 1 year ago

Hi @emmericp, can you help us with this issue?