Implement brotli everywhere

[asvetlov]

Now brotli is supported on client side for reading brotli compressed responses. We need to support it on other sides: sending client request, reading request on server and sending server response, multipart maybe.

See the difference between brotly and zlib usage. Ideally brotly should be supported everywhere we do support gzip by zlib library usage.

andrew•~/projects/aiohttp(master⚡)» pss zlib aiohttp                                               (aiohttp)  [11:58:30]
aiohttp/multipart.py
7:import zlib
391:            return zlib.decompress(data, -zlib.MAX_WBITS)
393:            return zlib.decompress(data, 16 + zlib.MAX_WBITS)
799:        zlib_mode = (16 + zlib.MAX_WBITS
800:                     if encoding == 'gzip' else -zlib.MAX_WBITS)
801:        self._compress = zlib.compressobj(wbits=zlib_mode)

aiohttp/http_parser.py
4:import zlib
622:            zlib_mode = (16 + zlib.MAX_WBITS
623:                         if encoding == 'gzip' else -zlib.MAX_WBITS)
624:            self.decompressor = zlib.decompressobj(wbits=zlib_mode)
635:                self.decompressor = zlib.decompressobj()

aiohttp/http_websocket.py
11:import zlib
184:                    # Compress wbit 8 does not support in zlib
198:                    # Compress wbit 8 does not support in zlib
218:    # compress wbit 8 does not support in zlib
221:                         'zlib does not support wbits=8')
282:                self._decompressobj = zlib.decompressobj(wbits=-zlib.MAX_WBITS)
547:                self._compressobj = zlib.compressobj(wbits=-self.compress)
551:                zlib.Z_FULL_FLUSH if self.notakeover else zlib.Z_SYNC_FLUSH)

aiohttp/web_response.py
7:import zlib
620:            zlib_mode = (16 + zlib.MAX_WBITS
621:                         if coding.value == 'gzip' else -zlib.MAX_WBITS)
622:            compressobj = zlib.compressobj(wbits=zlib_mode)

aiohttp/http_writer.py
5:import zlib
180:        zlib_mode = (16 + zlib.MAX_WBITS
181:                     if encoding == 'gzip' else -zlib.MAX_WBITS)
182:        self._compress = zlib.compressobj(wbits=zlib_mode)

andrew•~/projects/aiohttp(master⚡)» pss brotli aiohttp                                             (aiohttp)  [11:59:42]
aiohttp/http_parser.py
21:    import brotli
618:                    'Can not decode content-encoding: brotli (br). '
619:                    'Please install `brotlipy`')
620:            self.decompressor = brotli.Decompressor()

[fafhrd91]

btw I am not sure if zlib.compressobject does crc on gzip stream

[andrey-git]

Is it a good idea to brotli-compress dynamic content in runtime?

[asvetlov]

Maybe, but if should be off by default and controlled by user. Like we already do for gzip. Also server should compress only if Accept-Encoding contains br.

[fafhrd91]

From what I found it should faster and smaller than gzip on 4-5 compression levels

[asvetlov]

Done

[steverep]

Should this be closed? The linked PR was never merged and I don't see any documentation regarding the server supporting Brotli.

[steverep]

Thanks. Can the old PR be revived? If not, an incremental change to at least serve *.br files when available would be a big help.

[Dreamsorcerer]

It was closed by the author, you are welcome to fork their branch and open a new one.