search

madler / zlib

A massively spiffy yet delicately unobtrusive compression library.
http://zlib.net/
Other
5.46k stars 2.41k forks source link

Account for a forced stored block in deflateUsed() #989

Closed dridi closed 1 week ago

dridi commented 1 week ago

See https://github.com/madler/zlib/issues/189#issuecomment-2205172650 and https://github.com/varnishcache/varnish-cache/pull/3873#issuecomment-2205637791

With deflateUsed() as it currently stands, I have not been able to make all of our use cases work in Varnish Cache without poking at the deflate state. With this patch I can reliably collect the last bit position of the last deflate block in a GZIP stream, as far as our somewhat extensive test suite goes.

It also matched the last bit position collected by the fork we currently rely on with a corpus of more than 170000 files.

dridi commented 1 week ago

I have not been able to make all of our use cases work in Varnish Cache without poking at the deflate state

I had this one on my mind and I eventually realized that I may not be able to get the GZIP level at the end of the compression process, but I can definitely get it at the beginning. So I changed my patch to reflect that in the deflateUsed() documentation.

dridi commented 1 week ago

I suppose I'll need to try compressing my corpus again with different compression levels including zero to be sure.

dridi commented 1 week ago

Fixed in https://github.com/madler/zlib/commit/884e0c0809e4cc5e88d6545d1c49de7946c1b1ca and https://github.com/madler/zlib/commit/3adaa095a7a6c3f2cc00f73311f9bc8688c21aa7.