Add Datagram support to perf_client / perf_server

[stormshield-damiend]

Hi,

At stormshield we are currently looking at Quic Datagram as a transport protocol for tunneling packets. If you are interested we could wrote and propose a PR to add support for Datagram in perf_client and perf_server allowing benchmarking Stream or Datagram or a mix of both. We also plan to look at using a rate limiter (using governor crate) to allow sending Quic Datagram without bursting the physical link causing many packets drops, if you want we can also submit a PR on that topic.

[Ralith]

If you are interested we could wrote and propose a PR to add support for Datagram in perf_client and perf_server allowing benchmarking Stream or Datagram or a mix of both.

Sure, thanks!

We also plan to look at using a rate limiter (using governor crate) to allow sending Quic Datagram without bursting the physical link causing many packets drops

The intention is for this to be handled by the pacer. Are you seeing unreasonably large bursts in practice?

[stormshield-damiend]

Hi,

i made a proof of concept for this task on the following branch https://github.com/stormshield-damiend/quinn/tree/perf_datagram

The client and server use a reliable stream to send/receive the amount of data client will upload and also the amount of data received by the server. This allow both of them to synchronize when starting sending or reading datagrams and also count the amount of dropped frames.

I did some test on a loopback and it resulted in many dropped datagrams. The pacer does not seem to work well in this case.

I also modified the stats to display Mbits/sec as it the most commonly used units for bandwidth related applications.

Feel free to comment on the code.

I also did some tests using governor crate as upload rate limiter (not commited) but it is not working well as as soon some frames are dropped the bandwidth stats are not accurate.

[Ralith]

It looks like you're sending datagrams as fast as possible? That will always lead to many dropped datagrams; it's the intended behavior. The purpose of pacing is to avoid overwhelming the socket, not to avoid datagram loss. Reliable communication requires flow control, which datagrams by design don't have. Likely many datagrams are being dropped inside Quinn before they're even sent, exactly because pacing and congestion control are limiting the outgoing packet rate.

A better approach would be allow a desired data rate (and perhaps batch size) to be specified, then test it with rates below the link's capacity.

[stormshield-damiend]

Hi Ralith,

indeed it drops as we try to send as fast as we can, we know this, it is just a simple PoC.

We tried to limit packet rate using governor crate, but currently we have no way (API side) to know that quinn dropped the packet within send_datagram() due to send buffer size overflow, there is no proper way to reset the leaky bucket of the rate limiter in case of drop.

Do you think such API could be implemented to allow datagram users to implement their own rate limiter on application side ?

[Ralith]

That would have limited usefulness because in practice, the local QUIC stack's buffer is not the bottleneck; you should be far more concerned about packets being dropped by a router halfway down the network path than locally. There's been some discussion about providing feedback when a remote endpoint acknowledges a packet, but this is also difficult to use correctly, because the remote endpoint might drop it from its buffer before the remote application can read it.

The most reliable strategy is to implement application-layer acknowledgements. Of course, that rapidly starts looking like a half-baked re-implementation of streams, which should usually be preferred over datagrams if you need to send large quantities of data reliably.

But those are all questions of application design. For testing/benchmarking, we should focus on simple fixed data rates, which require no feedback regarding anything but the passage of time.