search

adamreichold / zeptohttpc

Proving ground for changes to attohttpc
Apache License 2.0
11 stars 0 forks source link

Fails with "Error: invalid gzip header" on 40% of websites #3

Closed Shnatsel closed 3 years ago

Shnatsel commented 3 years ago

On some websites, e.g. http://notesinspanish.com, zeptohttpc v0.2.1 fails with the following error:

Error: invalid gzip header

Firefox and curl work fine.

409,926 websites out of the top million from Feb 3 Tranco list are affected.

Tested using this code. Test tool output from all affected websites is too big to upload to Github, so here's a list of affected domains instead: invalid-gzip-header-domains.txt.gz

adamreichold commented 3 years ago

Thanks for testing. Decompression based on Content-Encoding was instantiated in the I/O stack were decompression based on Transfer-Encoding would be. Dropped the seldomly used Transfer-Encoding variant and fixed decompression to happen after chunk assembly.