search

dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
15.44k stars 4.76k forks source link

HTTP/3: QPack dynamic compression and decompression #49561

Open JamesNK opened 3 years ago

JamesNK commented 3 years ago

I believe QPackEncoder and QPackDecoder currently only support static header compression.

.NET should support dynamic compression to reduce network traffic and potentially save CPU usage spent encoding/decoding uncompressed header values.

ghost commented 3 years ago

Tagging subscribers to this area: @dotnet/ncl See info in area-owners.md if you want to be subscribed.

Issue Details
I believe QPackEncoder and QPackDecoder currently only support static header compression. .NET should support dynamic compression to reduce network traffic and potentially save CPU usage spent encoding/decoding uncompressed header values.
Author: JamesNK
Assignees: -
Labels: `area-System.Net.Http`
Milestone: -
geoffkizer commented 3 years ago

Do you have any specific scenarios in mind, or is this just a general request?

Dynamic compression/decompression is a good feature; the challenge here, as always, is to use it in the most effective way,

JamesNK commented 3 years ago

General request. It is a compete thing. When people compare Kestrel/HttpClient to other libraries they'll see "Hello world" REST or gRPC requests and responses be X bytes bigger on .NET because there is no dynamic compression.

When I added dynamic compression to Kestrel responses it not only reduced network traffic, it saved CPU. It is faster to lookup an entry in the dynamic table and write a small ID to the response than it was to encode and write the whole name and value. This was true even for the small response header values used by gRPC.

The overall benefit is variable depending on how much headers change from request to request, but from what I've seen dynamic compression is an overall perf win in network usage and RPS.

JamesNK commented 3 years ago

The HPack dynamic support in .NET is already pretty good. Both handling incoming dynamic headers (HttpClient/Kestrel) and sending outgoing dynamic headers (Kestrel).

I haven't looked closely at QPack at all. First questions:

  1. How different is QPack from HPack?
  2. How much of our HPack implementation and tests can we reuse?
geoffkizer commented 3 years ago

QPack is not that different than HPack in theory.

The main difference, as I understand it, is that HPack relies on HEADER blocks being processed in order, which suffers from head-of-line blocking. QPack defines a model that does not assume in order processing. As such it is somewhat more complex, but more flexible as well.

The static table is a bit different too, I think, but the difference is not huge.

karelz commented 3 years ago

Triage: Moving to Future, does not seem to be critical for 6.0. Note: We don't do compression for HPACK as it was too CPU intensive.

karelz commented 3 years ago

Triage: Leave it in Future for now. We can revisit if perf analysis demonstrates there may be measurable values (it did not for HTTP/2 HPACK). This is fairly isolated and should be straightforward spec implementation, marking as up-for-graps.