sanic-org / sanic

Accelerate your web app development | Build fast. Run fast.
https://sanic.dev
MIT License
18.08k stars 1.55k forks source link

Native Gzip & Brotli compression #2927

Open robd003 opened 7 months ago

robd003 commented 7 months ago

Is there an existing issue for this?

Is your feature request related to a problem? Please describe.

When serving larger responses it would be faster to send a compressed response as long as the client supports it.

Describe the solution you'd like

When a response from Sanic is over a certain threshold (2kb for Brotli / 512 bytes - 1kb for Gzip) and the HTTP client's accept headers permit it choose either Brotli or Gzip to compress the response

Additional context

It would be great to do this natively in Sanic rather than having to rely on something like nginx

Tronic commented 7 months ago

I am not against this but there are complications. First of all, the compression is not async but would need extra threads to avoid blocking the server. Brotli would need to be an additional (optional) dependency, while gzip is available in the standard library. Checking and doing this for every request also costs, possibly more than the gain (even just checking/setting the headers and of course the compression itself with slow CPU and fast network).

Ideally static data (files or other common responses) should be precompressed - I wrote my own static file handler to replace Sanic's for this (it also keeps all static files and their compressed versions in RAM - loaded and compressed at each server startup to avoid storing .br on disk, because a few megabytes of RAM "wasted" don't cost anything).

Dynamic data compression per request can also be useful for large JSON data and such, and if added as a feature in Sanic, and this of course should not be done in case apps or other facilities already do compression. Streaming responses would be difficult to support.

Both types of compression can also be handled very efficiently by a proxy such as Nginx or Caddy. At least the latter supports static file and dynamic compression in gzip, brotli and zstd formats with a simple config directive.

ahopkins commented 7 months ago

Are you talking about dynamic content or precomputing static content?

robd003 commented 7 months ago

This was for dynamic content.

@Tronic any chance you published your pre-compressed static file handler? That would be really useful!

dmckeone commented 2 months ago

If it helps at all, there is an open issue in sanic-ext for this: https://github.com/sanic-org/sanic-ext/issues/230

It includes example code for Sanic extension that tried to serve as a baseline, derived from older extensions out in the wild.

robd003 commented 2 months ago

Thanks!

Link to @dmckeone's extension: https://gist.github.com/dmckeone/b6d2160389640270eb24d4b0555a7768