Timestamp accuracy in captured pcap

[isaac-ped]

Hi,

I am attempting to write captured packets to a pcap file at high rate and with accurate timestamps.

I have based my script on the dump-pkts script, and found the suggestion to increase rxDescs and set dropEnable = false in this issue.

My script is here: https://gist.github.com/isaac-ped/c3154d6773a8a2fe799866cdffb8427e, and is currently being run using a single thread.

I am analyzing the captured pcap offline to determine the latency between requests and responses, and have found that (based on the recorded values) responses are often received before the requests are sent.

Using the batchTime = lm.getTime() approach did not work for me, as requests and responses would often be received in the same batch (expected latencies are < 10 us), so I attempted to use rxQueue:recvWithTimestamps(bufs) to get a unique timestamp for each packet.

Is there a better way to get precise timestamps in pcap dumps?

[emmericp]

recvWithTimestamps is the best possible implementation of software timestamps: a tight loop calling RDTSC. The problem you will run into here is batching by the NIC itself; you could try to adjust the descriptor write treshholds but that's messy. We currently use the defaults from DPDK: https://github.com/libmoon/libmoon/blob/master/src/device.c#L158-L162 Refer to the datasheet for details on what these values mean and try to adjust them.

That may or may not help. But there is really no good/reliable way to get precise/accurate timestamps in software at the low microsecond level; you'll have to use hardware timestamping if you need that.

Examples of NICs supporting hardware timestamps of all packets are the Intel X550 (10GBASE-T) and X552/Xeon D embdded NIC (available with SFP+).

We've got a setup with fiber optical splitters and an X552/Xeon D for timestamping all packets, achieving a ~20ns precision for latency measurements. The scripts we use for that were merged here: https://github.com/emmericp/MoonGen/pull/227/files

[isaac-ped]

Thanks for your quick response. We will likely move to the X552/Xeon D for timestamping purposes, if that is the recommended NIC for receiving timestamps with low-microsecond precision.

Alternately, if there is another recipe you could recommend for setting up the system to receive packet capture with accurate timestamping for offline analysis, we would be happy to hear it.

I would like to ask one more quick question regarding the dump-pkts.lua script that I based my capture.lua script on: is the memory buffer into which the packets are placed preallocated, or does it grow as more packets are written? If the latter, is there any way in which to force preallocation of the buffer?

[emmericp]

it's jsut a normale DPDK memory pool, so pre-allocated

[isaac-ped]

One last question: I am aware that the README states support for intel NICs. I was wondering if any or all of the following specific models have been shown to work well with MoonGen's ability to get hardware timestamp information from packets: (10G) 82559, X710, and (40G) XL710

[emmericp]

Full support for all of these and we use them a lot.

But none of them can timestamp all packets.

Am 19.03.2019 um 20:20 schrieb Isaac Pedisich notifications@github.com:

One last question: I am aware that the README states support for intel NICs. I was wondering if any or all of the following specific models have been shown to work well with MoonGen's ability to get hardware timestamp information from packets: (10G) 82559, X710, and (40G) XL710

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/emmericp/MoonGen/issues/238#issuecomment-474537367, or mute the thread https://github.com/notifications/unsubscribe-auth/AIhlAXUaoSSU5dDAv5qFg0XtaNnH0xNRks5vYTh_gaJpZM4b59bl.

-- Chair of Network Architectures and Services Department of Informatics Technical University of Munich Boltzmannstr. 3 85748 Garching bei München, Germany

[isaac-ped]

I see, thank you. We are attempting to determine the latency between requests and responses (over UDP) of a system under heavy load. If we randomly sample packets for timestamping, it's highly likely that we will miss one of either the request or response. Currently, we are attempting to capture all packets and process them offline, which requires timestamping all packets.

Other than acquiring a NIC which can timestamp all packets, is there a method to perform what we are trying to do? If not, are the Intel X550 (10GBASE-T) and X552/Xeon D the best options for the NIC?

[emmericp]

no