Add support for zstd decoding

[mbeijen]

Summary

This adds support for zstd decoding using the python package zstandard. This is similar to how it is implemented in urllib3. I also chose the optional installation option httpx[zstd] to mimic the same option in urllib3.

zstd decoding is similar to brotli, but in benchmarks it is supposed to be even faster. The zstd compression is described in RFC 8878.

See https://github.com/encode/httpx/discussions/1986

Checklist

• [x] I understand that this PR may be closed in case there was no previous discussion. (This doesn't apply to typos!)
• [x] I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
• [x] I've updated the documentation accordingly.

[tomchristie]

This is a lovely bit of functionality. Missing some coverage at the moment.

[mbeijen]

This is a lovely bit of functionality. Missing some coverage at the moment.

Thanks. I will see if I can add tests for full coverage

[mbeijen]

I've added full test coverage now, again by borrowing a test inspired from urllib3 @tomchristie please consider my PR again

[tomchristie]

please consider my PR again

Sure thing. (thanks)

• Are we crediting the urllib3 team sufficiently/appropriately here, perhaps a code comment in places where we're drawing from their work?
• Could you provide an example URL in this thread to test against?
• Let's get a CHANGELOG.md entry added.

[mbeijen]

• Are we crediting the urllib3 team sufficiently/appropriately here, perhaps a code comment in places where we're drawing from their work?

I've added a code comment in the appropriate places. In the docs and README there is already a credit given to urllib3 as a source for much of the lower level details, this is not much different. It is fair to add some extra credit here as well as a pointer for a reader to check the urllib3 implementatition in case of changes.

• Could you provide an example URL in this thread to test against?

Sure! Support for zstd is not really common on web sites yet. libcurl has support for some years now. Chrome has it now enabled by default so I will assume this might help? Also, Mozilla changed its stance on zstd from 'defer' to 'positive' in October, ref: https://github.com/mozilla/standards-positions/issues/775#issuecomment-1754635490 -- it is particularly well-suited for compressing JSON data.

Sites that support it include instagram.com, facebook.com, caddyserver.com

httpx -v --http2 https://caddyserver.com/  | grep encoding:
accept-encoding: gzip, deflate, br, zstd
content-encoding: zstd

• Let's get a CHANGELOG.md entry added.

I've added an entry.