search

chronoxor / CppServer

Ultra fast and low latency asynchronous socket server & client C++ library with support TCP, SSL, UDP, HTTP, HTTPS, WebSocket protocols and 10K connections problem solution
MIT License
1.43k stars 284 forks source link

Question: Why TCP echo benchmark so much faster than UDP echo benchmark? #36

Open nicodemuz-zz opened 4 years ago

nicodemuz-zz commented 4 years ago

From your examples in the Readme, the TCP benchmar pushes on the order 40 times more messages per second than UDP. Even SSL is on the order 10x faster than UDP.

When I compiled and ran it myself I got over 100 times more messages with TCP than what I got with UDP.

Why TCP and SSL echo benchmarks so much faster than UDP echo benchmark?

ne0phyte70 commented 3 years ago

Yes. I have also found this to be very surprising. Can the developer please comment? TIA

chronoxor commented 3 years ago

In TCP a stream buffer is used to put all outgoing messages together and send them in a batch with full IP packet. Batch mode is more effective.

In UDP a small datagram is send each time in a new IP packet which is not full.

It's just a kind of scenario. In other scenarios UDP could be more effective (e.g. broadcasting).

ne0phyte70 commented 3 years ago

Thx for your reply. Wouldn't this be an apples to oranges comparison then?  Shouldn't the stream buffer be used (or not used) in both cases to get a fair comparison?  Thx. Sent from Yahoo Mail on Android

On Sat, 9 Jan 2021 at 0:37, Ivan Shynkarenkanotifications@github.com wrote:

In TCP a stream buffer is used to put all outgoing messages together and send them in a batch with full IP packet. Batch mode is more effective.

In UDP a small datagram is send each time in a new IP package which is not full.

It's just a kind of scenario. In other scenarios UDP could be more effective (e.g. broadcasting).

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

chronoxor commented 3 years ago

Yes, but in those time for us TCP/SSL sessions were more important, so I add analogical UDP benchmark just for brief comparison. I'll think it over, maybe in near future we'll create benchmarking tests special for UDP.

ne0phyte70 commented 3 years ago

Thx for yr clarification.

Sent from Yahoo Mail on Android

On Sat, 9 Jan 2021 at 1:08, Ivan Shynkarenkanotifications@github.com wrote:

Yes, but in those time for us TCP/SSL sessions were more important, so I add analogical UDP benchmark just for brief comparison. I'll think it over, maybe in near future we'll create benchmarking tests special for UDP.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

nicodemuz-zz commented 3 years ago

With UDP packets fairly large, and with the benefit of a smaller header, the free´d capacity for more "packets" in batch mode doesn't explain a factor 100 in performance difference unless we go really close to 0 in packet size (payload size), or unless it is CPU limited and not throughput limited.

For 1 Gbps throughput limit, which should not be limit with test in same PC, a UDP packet totalling ~120 bytes including IP and UDP header should result in 1 million packets per second as each is ~1000 bits in size. But I am not getting close to those figures with UDP.

So is it a CPU constraint from CPU overhead at each packet transmission? I never did check the CPU usage...

By the way: Appropriate send and receive buffer sizes on TCP could even things out, and disabling Nagle algorithm would also adress the stream vs datagram issues making UDP and TCP implementation more comparable.