Using AF-XDP to increase packet ingestion through QUIC

[talalim]

Problem

When a validator becomes a leader, there's a significant drop in TPS as numerous clients attempt to establish connections. This decline stems from Quinn's limited capacity to manage active connections simultaneously, leading to swift evictions of older connections. Moreover, connection retries exacerbate the issue.

Proposed Solution

To enhance packet ingestion throughput, we propose leveraging AF_XDP to retrieve raw packets to userspace and implementing a lightweight, robust UDP stack, along with a QUIC library such as tquic.

AF_XDP circumvents traditional network stack overhead, resulting in reduced latency and improved throughput.

Additionally, to accommodate the increased throughput from userspace packet processing, it may be necessary to introduce multithreading to the sigverify stage, ensuring prompt transaction passage to execution.

[Huzaifa696]

I think tquic covers the main checkmarks for a good candidate QUIC library e.g

1. It must be well maintained by a well reputed organization
2. It must have better performance - benchmark results
3. It provides RUST APIs

Another alternative might be msquic which also covers the above criterion. Some other differentiating factors are:

• It is primarily written in C but provides the RUST API
• Generally there seems to be more dev activity (i.e no of active developers, no of commits per week etc) than tquic.
• It has better documentation