[Proposal] Use QUIC as experimental transport for internal gRPC server

ItalyPaleAle commented 1 year ago

In what area(s)?

/area runtime

Describe the proposal

The internal gRPC server is used by Dapr sidecars to communicate with each other during service invocation, and it currently uses "regular" gRPC, which relies on TCP-based HTTP/2.

One possible area of improvement could be to offer support for QUIC as an additional transport layer for this channel. This, which will initially be experimental, is meant to be deployed side-by-side with the existing HTTP/2-based transport.

QUIC is now an IETF standard (RFC9000) and it's underpinning HTTP/3. It offers some really nice benefits:

As the name implies, it's faster :) It doesn't suffer from head-of-line blocking issues like HTTP/2. It should also allow for lower latencies and possibly faster handshakes.
QUIC requires using TLS 1.3, which offers a better secure-by-default experience.
Because QUIC is based on UDP, it could enable some interesting scenarios in the future, such as NAT traversal.

POC

Here's a very rough POC demonstrating how QUIC could be used for the Dapr internal gRPC server. It works, although I have only done some very simple tests at this point (e.g. I haven't tested load-balancing across multiple instances, and I suspect that may require more work).

The POC is possible thanks to forking sssgun/grpc-quic.

Caveats

The first thing to keep in mind is that the gRPC library doesn't support HTTP/3 or QUIC natively, at least not for Go. There's a design proposal for HTTP/3 transport in gRPC which has been approved almost 2 years ago, and that's been implemented in the .NET library. However, the Go library for gRPC doesn't yet have support for HTTP/3 or QUIC.
For Dapr to support QUIC, we'd need to implement our own transport. This is not overly complex (in fact, see my POC above), but it's not built into the upstream library.
Also important to note that QUIC (as in the IETF standard) is not HTTP/3. In fact, QUIC is a transport protocol, and HTTP/3 is based on top of that. The way the POC uses QUIC is more akin to how we use Unix sockets for gRPC, in fact.
The Go gRPC library is going to add support for HTTP/3, but not QUIC directly, as the higher-level protocol has some benefits as they explain in the linked issue.
The biggest consequence of using QUIC directly rather than HTTP/3 is that it would be a "custom protocol" and not understood by things like proxies, should users need to rely on them.
One more caveat is that QUIC requires TLS encryption, so in Dapr it can only be used if mTLS enabled in the cluster.
Lastly, using QUIC (and that will include also HTTP/3) currently requires a 3rd-party library quic-go. This library is used by dozens of very large apps (including traefik, caddy, cloudflared, and ipfs), but because of how it intertwines with the Go standard library, it requires a new update every time a new version of Go comes out. This means that when we update Go, we also need to update quic-go, and users on a new version of the Go compiler won't be able to build Dapr until we do that.

Considerations

Given the above caveats, a possibility could be to just wait for go-grpc to add support for HTTP/3 natively. This would be ideal, and it will happen, but at this point it's unclear when that will happen as it doesn't seem like it's being prioritized. If we were to move forward with offering QUIC in Dapr today, we should consider that as an experimental, preview feature for now.

dapr-bot commented 1 year ago

This issue has been automatically marked as stale because it has not had activity in the last 60 days. It will be closed in the next 7 days unless it is tagged (pinned, good first issue, help wanted or triaged/resolved) or other activity occurs. Thank you for your contributions.

dapr-bot commented 1 year ago

This issue has been automatically closed because it has not had activity in the last 67 days. If this issue is still valid, please ping a maintainer and ask them to label it as pinned, good first issue, help wanted or triaged/resolved. Thank you for your contributions.

dapr / dapr