search

erpc-io / eRPC

Efficient RPCs for datacenter networks
https://erpc.io/
Other
835 stars 137 forks source link

Question: verbs in raw transport #56

Closed RicardoDZX closed 3 years ago

RicardoDZX commented 3 years ago

Hi, i read your paper in NSDI 19 and roughly go through the source code. It is a great work, but i am not very familiar with RDMA-programming and have some confusions about it.

In the paper, "eRPC works well with UDP packets over lossy Ethernet". I read the implementation of "raw transport" and find that eRPC does not leverage linux socket but RDMA ( libibverbs? ) verbs (ibv_post_send, \etc.) to transfer the constructed UDP packets. Is it right?

Then, we are curious about: is that function general/ can be used in normal / RDMA-incapable devices? (or maybe most of modern commercial NICs/OS support verbs?)

Thanks

RicardoDZX commented 3 years ago

And another question: It seems that the current version of TIMELY implemented in eRPC does not leverage NIC hardware timestamp to calculate RTT. or i miss some details in code?

RicardoDZX commented 3 years ago

I read some docs about DPDK and RDMA, and also NIC support in eRPC's readme. For now, i understand that the method used in eRPC is : DPDK for Ethernet NIC+ raw using RDMA verbs for RDMA-capable NIC + Infiniband for RDMA-capacle NIC and RDMA-capable network.

Therefore, DPDK-capable NIC satisfies the minimum requirements. RDMA-capable NIC performs better (with RDMA verbs)\etc?

But still know little about whether eRPC uses the NIC hardware timestamps in TIMELY or not.

anujkaliaiitd commented 3 years ago

Hi Ricardo. Thanks for your interest.

eRPC does not use NIC hardware timestamps for now. Although hardware timestamps are better suited for Timely, they are tricky to configure (e.g., we need to synchronize the host clock with the NIC clock), especially across multiple hardware and software platforms.

RicardoDZX commented 3 years ago

Thanks for your reply!