search

ido / libvma-old

Automatically exported from code.google.com/p/libvma
Other
0 stars 1 forks source link

Add receive software packet timestampping support #7

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
These patches update libvma to support software receive packet timestampping, 
with SO_TIMESTAMP, SO_TIMESTAMPNS, and SO_TIMESTAMPING.

Additionally I'll mention that I really would like to have hardware packet 
timestamps but it is my understanding that the infrastructure to get and 
synchronize the hardware timestamps with the system time is still in 
development.

Original issue reported on code.google.com by shawn.bo...@gmail.com on 29 Jan 2014 at 7:32

Attachments:

GoogleCodeExporter commented 9 years ago 
Hi Shawn,

Thanks for the patches.

I pushed the first one, but i have a two questions about the second one:
a. To my understanding the timestamp need to be taken when the packet is 
arriving to the interface from the NIC, and not when the user call the 
recvmsg() function. I think the linux kernel takes the timestamp even before 
the upper protocols (ip/tcp), and save it in the skb. I will move the timestamp 
taking to rx_input_cb(), and store it in our mem_buf_desc_t (skb analogous). 
Let me know if you think it should behave differently.
b. Do you have a special reason to use CLOCK_REALTIME?
Why not CLOCK_MONOTONIC or RDTSC?
If you have none, i would like to change it to RDTSC (see 
./src/vma/util/rdtsc.h).

Regarding HW timestamps, this feature is available in our NIC and driver, but 
is not fully exposed to user-space through verbs yet. Once it will, we will add 
support for it in libvma.

Original comment by orkmella...@gmail.com on 1 Feb 2014 at 8:59

GoogleCodeExporter commented 9 years ago 
a) You are totally correct and I should have mentioned that I probably wasn't 
grabbing the timestamp in the best place.  I'm slowly working my way through 
the libvma code and wasn't sure where the earliest place VMA sees the packet.  
Feel free to grab the time at the earliest place you can.  Like you mentioned 
in the kernel the skb is normally timestampped in netif_receive_skb() which is 
the earliest point that the packet enters the networking core.  At that point 
the kernel timestamps the skb with ktime_get_real() which leads to...

b) The ktime_get_real() timestamp on the skb is used for SO_TIMESTAMP, 
SO_TIMESTAMPNS, and the software timestamp of SO_TIMESTAMPING.  
ktime_get_real() is equivalent to CLOCK_REALTIME.  Aside from simply matching 
the kernel's behavior, our use case typically uses the packet timestamp to 
compute delays through our system and even across machines.  We synchronize our 
system clocks with PTP and thus we can even  compute one-way delays with 
reasonable accuracy of packets between machines.

Original comment by shawn.bo...@gmail.com on 2 Feb 2014 at 12:15

GoogleCodeExporter commented 9 years ago 
I moved the timestamp taking to when the packet reach the socket.
It can be a done a little earlier, when the packet is pulled from the CQ, but 
it is much more convenient to do it in the socket.

I stayed with the CLOCK_REALTIME.

https://code.google.com/p/libvma/source/detail?r=f6c8b49c677ef5463b438a25a42624b
a7d4653cd

https://code.google.com/p/libvma/source/detail?r=517b387b6d76d5f9d75df923f071696
24773d79c

https://code.google.com/p/libvma/source/detail?r=4368d99de8ef4b8086309cbbcd814a0
a56ee5777

Original comment by orkmella...@gmail.com on 2 Feb 2014 at 11:57

GoogleCodeExporter commented 9 years ago 
    1599    +   if (m_b_rcvtstamp || (m_n_tsing_flags & (SOF_TIMESTAMPING_RX_SOFTWARE | SOF_TIMESTAMPING_SOFTWARE))) {
    1600    +   memset(&(p_desc->path.rx.timestamp), 0, sizeof(struct timespec));
    1601    +   clock_gettime(CLOCK_REALTIME, &(p_desc->path.rx.timestamp));
    1602    +   }

Is there reason for the memset before calling clock_gettime?  Are you afraid 
clock_gettime() might fail leaving garbage in the timestamp?

Original comment by shawn.bo...@gmail.com on 3 Feb 2014 at 11:35

GoogleCodeExporter commented 9 years ago 
I removed the memset, but added another initializtion on mem_buff_desc recycle.

please see 
https://code.google.com/p/libvma/source/detail?r=364f41da4111a64fb22364d84982d31
c0c630849

Original comment by orkmella...@gmail.com on 6 Feb 2014 at 10:32

GoogleCodeExporter commented 9 years ago 
Your latest change looks good to me, and I would also uncomment your TODO for 
when timestampping was requested after a packet has already arrived since that 
is exactly what the kernel does.  See __sock_recv_timestamp() in net/socket.c.

Original comment by shawn.bo...@gmail.com on 6 Feb 2014 at 5:16

GoogleCodeExporter commented 9 years ago 
I uncomment the TODO.
I also added a memset to 0 for tsing, as we were returning garbage in the HW 
info for tsing_flags.

https://code.google.com/p/libvma/source/detail?r=f5e064e8c5986014f69a65fdbbfe0da
680bfd06d

Original comment by orkmella...@gmail.com on 6 Feb 2014 at 6:48