search

erpc-io / eRPC

Efficient RPCs for datacenter networks
Other
851 stars 138 forks source link

Why UDP as the transport layer protocol? Can UDP have some performance gains compared with http2/TCP? #60

Closed RicardoDZX closed 3 years ago

RicardoDZX commented 3 years ago

Hi, i have some questions about transport layer design in the paper.

I understand that eRPC was designed to achieve a comparable performance against RDMA \etc(in datacenter networks). However, as a RPC framework, i am curious about the performance of eRPC over other RPC library :

Kernel-bypass and the msgbuf optimizations may reduce latency & increase the RPS performance. However, are there some performance gains origin from the selection of transport layer protocol (UDP)? For instance: (1) i observe that gRPC uses http2/TCP and eRPC uses UDP. I am curious about whether there exist some performance gains by using UDP + congestion control(rather than TCP) as the transport layer protocol or not. Could you give some intuitions about eRPC's benefits to modern commodity RPC library(gRPC, \etc). (Maybe the overhead to maintain a TCP connection?)

(2) I saw in the comments of the code you say "A session is a connection between two eRPC endpoints (similar to a

Look forward to your kind reply, thanks

RicardoDZX commented 3 years ago

I remember that using unreliable protocols(UDP) can reduce the number of network messages(Section 3.5.1 in your paper Efficient Remote Procedure Calls for Datacenters)

Are there any other intuitions about using UDP as the transport layer protocols?

RicardoDZX commented 3 years ago

Feel sorry for bothering but another question is : terminate called after throwing an instance of 'std::runtime_error' what(): eRPC supports only mlx4 or mlx5 devices with DPDK, (use net_ixgbe). Aborted (The assert is coded in dpdk-transport.cc) I use Intel 82599 NIC, it seems that the current version of eRPC cannot support dpdk + intel dpdk-capable NICs? (only dpdk + mlx poll mode driver is supported?)

For clearly, in this issue: (1) the intuition to use UDP as the transport layer of a RPC framework. At present , the benefits i have learned are 'reduce network messages' and 'simplify the protocol'
(2) Is Current version of eRPC still support intel DPDK-capable NICs? or only MLX CX4 / CX5 are supported?

anujkaliaiitd commented 3 years ago

Thanks for your interest. Here are my thoughts:

RicardoDZX commented 3 years ago

OK，I see...... Thanks for your reply!

Recent days i have worked on deploying eRPC in a container. As mentioned above, ENV: Inter 82599 + DPDK. I use SRIOV to pass through VF to the container as DPDK phy_port to run eRPC. With some minor changes & just comments the assert above, the current version of eRPC still works! (\i.e, i successfully run the helloworld and latency app) The only point is that the latency performance seems worse than that claimed in the NSDI paper. (about 80 μs )

Perhaps it is because that DPDK performs worse than MLX / RDMA verbs? Would you like to provide me a baseline on eRPC's latency performance(under Intel DPDK-capable NICs tests)?

anujkaliaiitd commented 3 years ago

With DPDK on Mellanox CX3--CX5 NICs, eRPC can get 3--4 microsecond round trips. With Intel 82599 or XL710 NICs, I've measured round trip latency between 10 and 20 microseconds.

RicardoDZX commented 3 years ago

OK！ Thanks for your reply！！

RicardoDZX commented 3 years ago

Finally results of dpdk + Intel 82599 NIC +VF for containers: latency: median 6-8 microseconds.

Poor performance before (around 100 microseconds) may result from the LOG_LEVEL=trace. After tuning the LOG_LEVEL to warn, eRPC seems perform well.

anujkaliaiitd commented 3 years ago

Thanks for sharing your performance results. That latency is a fair bit lower than what I expected from 82599.

I will add net_ixgbe as a supported device.