search

quinn-rs / quinn

Async-friendly QUIC implementation in Rust
Apache License 2.0
3.76k stars 380 forks source link

Horizontally scalable endpoints #1578

Open Ralith opened 1 year ago

Ralith commented 1 year ago

Having a proper go at #1576. Reduces the inter-task communication required at the quinn layer by allowing connection tasks to manipulate the endpoint directly. Forms a strong foundation for a more incremental and performant attempt at #1219.

The high-level architecture is largely unchanged for now, but this is already enough for a hypothetical motivated quinn-proto user to implement good horizontal scaling.

This PR is organized into four major sections:

  1. Incidental cleanup/fixes
  2. Refactoring of mutable state inside proto::Endpoint
  3. Placement of remaining mutable endpoint state behind locks, allowing all methods to take &self.
  4. Refactoring proto::Connection to call directly into proto::Endpoint rather than relying on the caller (e.g. quinn) to pass messages

Next steps (probably in future PRs) might be to move ownership of proto::Connections into the proto::Endpoint, followed by finally attempting to unify the drivers again. We'll need to work out how to get multiple coarse endpoint drivers to play nicely with tokio before this can be completed.

djc commented 1 year ago

Maybe split off the incidental cleanups/fixes into a separate PR? Easier to get through, and less prone to change (as in your latest insights from #1576, which I assume might affect this PR). (Maybe also the refactoring of mutable state, not sure how much sense that makes on its own?)

Ralith commented 1 year ago

Yep, I'll try to split this up some. I'm indeed wary of committing to the direction explored here prematurely; in particular I don't intend that we merge any major redesigns to how work is distributed/tasks are synchronized without careful benchmarking demonstrating similar or better performance under a broad range of workloads. There's a risk of optimizing for one shiny hypothetical case at the cost of other more immediate ones.

Ralith commented 1 year ago

Regarding the interior mutability, I do like the simpler API and behavior it entails, but that comes at the cost of greater risk of contention under worst-case conditions, which an attacker might deliberately produce. Not sure if it'll pan out.