Open carwyn opened 7 years ago
This is a bug, if the content type is deflate it shouldn't be trying gzip it should use zlib. Note thought that there are some servers that are broken an use raw deflate format.
From: http://www.gzip.org/zlib/zlib_faq.html#faq38
"gzip" is the gzip format, and "deflate" is the zlib format. They should probably have called the second one "zlib" instead to avoid confusion with the raw deflate compressed data format. While the HTTP 1.1 RFC 2616 correctly points to the zlib specification in RFC 1950 for the "deflate" transfer encoding, there have been reports of servers and browsers that incorrectly produce or expect raw deflate data per the deflate specficiation in RFC 1951, most notably Microsoft.
Neither the Go or Python stdlib url/http handlers deal with the deflate debacle for you. The Python Requests lib does though and it really is a case of try one, fail, try the other (i.e. peek at the bytes).
The python Requests lib implementation is here
And this is a very useful test page http://carsten.codimi.de/gzip.yaws/
What the HTTP 1.1 standard has to say about this:
http://www.sueddeutsche.de