search

ipfs / infra

Tools and systems for the IPFS community
MIT License
133 stars 41 forks source link

Gateway: Support Brotli Compression #488

Closed lidel closed 2 years ago

lidel commented 5 years ago

Something I discovered when playing with http-compression-check.sh : when asking for the same HTML file (Vincent_van_Gogh.html) from ipfs.io and cloudflare-ipfs, the latter will consume less bandwidth:

ipfs.io/ipfs/

Checking https://ipfs.io/ipfs/QmT5NvUtoM5nWFfrQdVrFtvGfKFmG7AHE8P34isapyhCxX/wiki/Vincent_van_Gogh.html
Content-Encoding: gzip
compressed:     102115
uncompressed:   482198

cloudflare-ipfs.com/ipfs/

Checking https://cloudflare-ipfs.com/ipfs/QmT5NvUtoM5nWFfrQdVrFtvGfKFmG7AHE8P34isapyhCxX/wiki/Vincent_van_Gogh.html
Content-Encoding: br
compressed:      86049
uncompressed:   482198

I wonder, how gateway cpu/bandwidth utilization would change if we added support for brotli? Are we able to enable it on some gateways to evaluate the impact?

References

mburns commented 5 years ago

Are we able to enable it on some gateways to evaluate the impact?

We are. Looks like it will involve compiling a custom nginx build, but not too difficult. Alternatively, OpenResty (which we have briefly used in the past as a drop-in replacement for nginx) comes with Brotli pre-installed.

mburns commented 3 years ago

good and bad news.

Good news: It appears the brotli nginx module is under active-ish development again, after being in hiatus for a while.

Bad news: OpenResty no longer ships brotli support out of the box. We will have to build our own.

Although, asking them to include brolti support upstream is also an option.

mburns commented 3 years ago

status:

thattommyhall commented 2 years ago

https://docs.nginx.com/nginx/admin-guide/dynamic-modules/brotli/