Open mdickopp opened 2 years ago
Thanks for the detailed breakdown!
When I looked at the source code (to see if I can implement a fix), I found another case that is incorrect:
Command | Expected Encoding | Observed Encoding |
---|---|---|
curl -I -H 'accept-encoding: gzip;q=0.995, br;q=0.999' http://127.0.0.1:3000/ |
br | gzip |
There's another case:
Command | Expected Encoding | Observed Encoding |
---|---|---|
curl -I -H 'accept-encoding: *;q=0.8, gzip;q=0.5' http://127.0.0.1:3000/ |
no encoding (identity) or any encoding supported by the server other than gzip | gzip |
I will work on a fix for this issue.
@mdickopp Thanks for your work on https://github.com/tower-rs/tower-http/pull/220! Would you mind editing your comments here with the cases that aren't handled yet? Should make it easier for people to pick up and help.
Sure. Cases 4 and 6 are currently unresolved.
Another related issue:
grpc-web
relies on the normal Content-Encoding
headers for compression. There's a default predicate in CompressionLayer to avoid compressing application/grpc
requests which unfortunately also filters out application/grpc-web
requests because the predicate just checks if there's a prefix match:
This means I need to either remove this predicate or not have compression for grpc-web requests
Bug Report
Version
tower-http v0.2.1
Platform
Debian Linux 12 (“bookworm”) Linux feynman 5.15.0-3-amd64 #1 SMP Debian 5.15.15-1 (2022-01-18) x86_64 GNU/Linux rustc 1.58.1 (db9d1b20b 2022-01-20)
Description
When using
CompressionLayer
, theaccept-encoding
header sent by the client is not parsed correctly (i.e., according to RFC 7231, sections 5.3.1 and 5.3.4). The following program demonstrates the issues (using axum v0.4.5):Case 1: Uppercase encodings and qvalues are not parsed (fixed by #220)
Encodings and qvalues are case-insensitive, i.e. the server should understand them whether they are lowercase, uppercase, or a mixture of both.
curl -I -H 'accept-encoding: GZIP' http://127.0.0.1:3000/
curl -I -H 'accept-encoding: gZiP' http://127.0.0.1:3000/
curl -I -H 'accept-encoding: gzip;q=0.5, br;Q=0.8' http://127.0.0.1:3000/
Case 2: Spaces before and after semicolon are not parsed (fixed by #220)
Space and horizontal tab characters are allowed before and after the semicolon separating the encoding from the qvalue.
curl -I -H 'accept-encoding: gzip;q=0.5, br; q=0.8' http://127.0.0.1:3000/
curl -I -H 'accept-encoding: gzip;q=0.5, br ;q=0.8' http://127.0.0.1:3000/
curl -I -H 'accept-encoding: gzip;q=0.5, br ; q=0.8' http://127.0.0.1:3000/
Case 3: Invalid qvalues are accepted (fixed by #220)
Qvalues are expected to have exactly 1 digit before and not more than 3 digits after the decimal point.
curl -I -H 'accept-encoding: gzip;q=00.5' http://127.0.0.1:3000/
curl -I -H 'accept-encoding: gzip;q=0.5000' http://127.0.0.1:3000/
curl -I -H 'accept-encoding: gzip;q=.5' http://127.0.0.1:3000/
Case 4: Request not rejected if client rejects identity encoding
If the client explicitly rejects the identity encoding or the wildcard encoding
*
, and accepts no encodings supported by the server, the request should be rejected.curl -I -H 'accept-encoding: identity;q=0' http://127.0.0.1:3000/
curl -I -H 'accept-encoding: *;q=0' http://127.0.0.1:3000/