Closed andrew-aladev closed 4 years ago
I don't understand what you think the issue is.
chunked, gzip may look meaningless but it is possible and its behaviour is well described.
Yes, its behaviour is described:
If a Transfer-Encoding header field is present in a request and the chunked transfer coding is not the final encoding, the message body length cannot be determined reliably; the server MUST respond with the 400 (Bad Request) status code and then close the connection.
@sam-github, this exception is assigned for the message body length cannot be determined reliably
+ chunked transfer coding is not the final encoding
. It means that you can't receive HTTP header with Transfer-Encoding: chunked, gzip
and without Content-Length
.
But if Content-Length
exists than Transfer-Encoding: chunked, gzip
is valid header.
From the very same 3.3.3:
If a message is received with both a Transfer-Encoding and a
Content-Length header field, the Transfer-Encoding overrides the
Content-Length. Such a message might indicate an attempt to
perform request smuggling (Section 9.5) or response splitting
(Section 9.4) and **ought to be handled as an error**.
In other words, Transfer-Encoding cannot be present together with Content-Length.
It must be said that the presence of both headers leads to quite practical request smuggling attacks as was demonstrated by security researchers.
chunked
encoder/decoder should be shipped outside http parser as separate library. It should be used by end-user together with gzip, brotli, zstd, compress encoders/decoders.
This simply can't work as the presence of chunked
TE alters protocol in a way that cannot be offloaded to a 3rd party module.
Hello @indutny, I can't agree with you, please read carefully.
If a Transfer-Encoding header field is present in a response and the chunked transfer coding is not the final encoding, the message body length is determined by reading the connection until it is closed by the server.
If a Transfer-Encoding header field is present in a request and the chunked transfer coding is not the final encoding, the message body length cannot be determined reliably; the server MUST respond with the 400 (Bad Request) status code and then close the connection.
If a message is received with both a Transfer-Encoding and a Content-Length header field, the Transfer-Encoding overrides the Content-Length. Such a message might indicate an attempt to perform request smuggling.
3 exception is related to request only, response just reads as much as possible data from server and has no possible security exceptions.
Please read RFC 7230 - 3.3.1 Transfer-Encoding for clarification too.
If any transfer coding other than chunked is applied to a request payload body, the sender MUST apply chunked as the final transfer coding to ensure that the message is properly framed.
If any transfer coding other than chunked is applied to a response payload body, the sender MUST either apply chunked as the final transfer coding or terminate the message by closing the connection.
This simply can't work as the presence of chunked TE alters protocol in a way that cannot be offloaded to a 3rd party module.
I see no collisions. Transfer-Encoding: chunked, gzip
means just gzip after chunked. User are reading data, than applies gzip
and than chunked
.
I was wrong about Content-Length
: it can't be used to identify the length of body if Transfer-Encoding
was used, this usage is forbidden. But responses without Content-Length
can have Transfer-Encoding
with chunked
in any position with other encodings.
Well, FWIW it is allowed for responses: https://github.com/nodejs/http-parser/blob/5c5b3ac62662736de9e71640a8dc16da45b32503/http_parser.c#L1904-L1913
Parser doesn't depend on gzip
, brotli
, etc it can't decode chunked for user, because there is another encoding on the top of chunked. User have to handle chunked by himself anyway.
Hello. Please look in RFC 7230 - 3.3.3 Message Body Length
As I can understand it means that both
Transfer-Encoding: gzip, chunked
andTransfer-Encoding: chunked, gzip
are possible.chunked, gzip
may look meaningless but it is possible and its behaviour is well described.the message body length is determined by reading the connection until it is closed by the server
means that we need to decodegzip
first and thanchunked
.But it means that all
nginx
,mongrel
,nodejs
(and its descendants) http parsers are broken by design.chunked
encoder/decoder should be shipped outside http parser as separate library. It should be used by end-user together withgzip
,brotli
,zstd
,compress
encoders/decoders.Another solution is to develop fat parser with
gzip
,brotli
,zstd
,compress
, etc dependencies.This issue can't be fixed. I want just to inform everyone using http parser.