question netmap-libpcap timestamp

[GoogleCodeExporter]

First, I apologize for double posting for I've posted this question on 
netmap-libpcap. But I decided posted here as there are more contributors on 
this project email list.  

What steps will reproduce the problem?
1.Run tcpdump with netmap mode enabled (with high traffic 10G over 
ixgbe(withnetmap patch))
2.Run tcpdump without netmap (with high traffic 10 over ixgbe)
3.Compare timestamp, under high traffic, netmap mode captured packets exhibits 
signs of "batch timestamp"

What is the expected output? What do you see instead?
Expected timestamp is to be be per packet, it seems netmap is time stamping the 
entire ring buffer with the same value

What version of the product are you using? On what operating system?
OS: Ubuntu 14.04 64bit (3.13.0-36-generic)
netmap git HEAD: 3ccdadad7d80bb1bc569d1f72ba3ece902350f65
netmap-libpcap git HEAD: 0890b993dae48767af172b693f2d664181dbe943
tcpdump: version 4.5.1

Please provide any additional information below.

Below are captured at 10G rate (at 1500 bytes), below is tcpdump without netmap 
(shows micro-level differences between each packet)

14:26:37.475718 IP 192.85.1.4 > 193.85.1.4:  ip-proto-253 1462
14:26:37.475726 IP 192.85.1.4 > 193.85.1.4:  ip-proto-253 1462
14:26:37.475727 IP 192.85.1.4 > 193.85.1.4:  ip-proto-253 1462
14:26:37.475728 IP 192.85.1.4 > 193.85.1.4:  ip-proto-253 1462
14:26:37.475729 IP 192.85.1.4 > 193.85.1.4:  ip-proto-253 1462

However, on the netmap mode capture, groups of packets had the same timestamp

14:15:19.931624 IP 192.85.1.4 > 193.85.1.4:  ip-proto-253 1462
14:15:19.931624 IP 192.85.1.4 > 193.85.1.4:  ip-proto-253 1462
14:15:19.931624 IP 192.85.1.4 > 193.85.1.4:  ip-proto-253 1462
14:15:19.931624 IP 192.85.1.4 > 193.85.1.4:  ip-proto-253 1462
14:15:19.931624 IP 192.85.1.4 > 193.85.1.4:  ip-proto-253 1462

Examining the code, I found this under /bpf/net/netmap_user.h

#define D(_fmt, ...)                                            \
        do {                                                    \
                struct timeval t0;                              \
                gettimeofday(&t0, NULL);                        \
                fprintf(stderr, "%03d.%06d %s [%d] " _fmt "\n", \
                    (int)(t0.tv_sec % 1000), (int)t0.tv_usec,   \
                    __FUNCTION__, __LINE__, ##__VA_ARGS__);     \
        } while (0)

/* Rate limited version of "D", lps indicates how many per second */
#define RD(lps, format, ...)                                    \
    do {                                                        \
        static int t0, __cnt;                                   \
        struct timeval __xxts;                                  \
        gettimeofday(&__xxts, NULL);                            \
        if (t0 != __xxts.tv_sec) {                              \
            t0 = __xxts.tv_sec;                                 \
            __cnt = 0;                                          \
        }                                                       \
        if (__cnt++ < lps) {                                    \
            D(format, ##__VA_ARGS__);                           \
        }                                                       \
    } while (0)
#endif

It seems to be an intended design, but I wanted to check with the netmap team 
on this. The downside of batch timestamping is we are observing out of order 
packets. 

Thank you for looking into this and again I'm sorry for double posting

Original issue reported on code.google.com by morph...@gmail.com on 30 Sep 2014 at 7:19

[GoogleCodeExporter]

Yes batch timestamping is intended behaviour. Changing this a bit, by putting a 
timestamp in individual buffers as packets are processed (which does not 
correspond to the actual reception) would have some significant loss of 
performance at high rates, and no real advantage. Note that the software 
timestamping done in the network stack does not reflect arrival times either: 
even without netmap you get an interrupt from the NIC with a one or more 
packets, then the napi/softintr thread starts timestamping packets as it 
processes them.

The "out of order" packets (I assume you refer to packets from different 
queues) phenomenon could be observed also with the standard drivers.

On passing: the D(), RD() and ND() macros are generic debugging helper not 
related to
packet timestamps.

Original comment by rizzo.un...@gmail.com on 30 Sep 2014 at 7:41

• Changed state: Invalid
• Added labels: Priority-low, type-other
• Removed labels: Priority-Medium, Type-Defect

[GoogleCodeExporter]

Hi Rizzo:

Thank you for your quick response. What I meant by "out of order" purely from a 
post processing stand point. We are using netmap as a pcap logger. But post 
processing tools such as wireshark attempts to index packets based on 
timestamp, multiple packets sharing the same time timestamp would not get 
presented in accurate order in wireshark. I understand the lack of accuracy of 
software timestamping overall. Just out of curiousity, what is performance 
penalty of having timestamp on individual buffers? I understand much 
improvement has been put into netmap, is it still be faster than standard 
networking stack? 

Original comment by morph...@gmail.com on 30 Sep 2014 at 8:29

[GoogleCodeExporter]

The reordering is a bug in whatever presentation tool you are using. It should 
use a stable sort algorithm.
With a microsecond timestamp resolution there is no way netmap or any other 
capture library can guarantee that all packets have different timestamps (they 
can be 67ns apart on the wire).

Original comment by rizzo.un...@gmail.com on 1 Oct 2014 at 10:16