Open cancan101 opened 2 years ago
Thanks @cancan101 - looks like a case of h11
being (reasonably enough) stricter about erroring on a non-compliant response. However, really we would like to be able to be robust to this kind of thing.
Multiple Transfer-Encoding
headers are explicitly forbidden in specs. But browsers, and many libraries, accept them; so this is a case that we may have to deal with at some point.
To ease this restriction you can monkey-patch h11._headers.normalize_and_validate
.
def _patch(headers, _parsed=False):
...
import httpx
# monkey-patch AsyncClient
httpx._transports.default.httpcore._async.http11.h11._headers.normalize_and_validate = _patch
# monkey-patch Client? (not tested)
httpx._transports.default.httpcore._sync.http11.h11._headers.normalize_and_validate = _patch
{'transfer-encoding': 'chunked'}
headers.This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Boop
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
any update on a fix?
@logan-vitelity This is actually an h11
issue. It seems that they are open to loose the restriction, but for the time being, there are no changes.
At the moment the only solution is the monkeypatching.
Just a suggestion for someone who face the same problem:
## _headers.py line 89
if saw_transfer_encoding:
pass
# raise LocalProtocolError("multiple Transfer-Encoding headers",
# error_status_hint=501)
## _connection.py line 88
if transfer_encodings:
assert transfer_encodings == [b"chunked"] or b"chunked" in transfer_encodings
return ("chunked", ())
Multiple Transfer-Encoding headers are explicitly forbidden in specs. But browsers, and many libraries, accept them; so this is a case that we may have to deal with at some point.
RFC 7230 is superceded by RFC 9112.
I read RFC 9112 looking for the "explicitly forbidden" spec... I cannot find it. But I know that RFCs can often be vague to the point of having multiple mutually exclusive yet reasonable interpretations.
Can someone clarify where the "explicitly forbidden" language is located in the RFC?
@danstoner Okay, I've taken another look over the h11
code...
Given that h11
will gracefully ignore multiple Content-Length
headers (so long as they have the same value)...
It would be a reasonable change for it to also gracefully ignore multiple Transfer-Encoding
headers (so long as they have the value "chunked")
Would you consider a pull request on h11
for that change?
I'll also link to https://github.com/python-hyper/h11/issues/175 so that if anyone is minded to start on a release we can hopefully get this resolved.
Running:
produces
RemoteProtocolError: multiple Transfer-Encoding headers
.Using
requests
, works fine: