Use zlib-ng instead of zlib

[nulano]

Alternative to #8495 .

Changes proposed in this pull request:

• Replace zlib with zlib-ng built with CMake in Windows builds. Building our own zlib-ng allows us to use the latest release since the zlib-ng 2.2.2 release is missing the x86_64 Windows build.

• Add a flag to PIL.features for zlib-ng. python3 -m PIL.report now looks like this:

  --------------------------------------------------------------------
  Pillow 11.1.0.dev0
  Python 3.11.9 (tags/v3.11.9:de54cf5, Apr  2 2024, 10:12:12) [MSC v.1938 64 bit (AMD64)]
  --------------------------------------------------------------------
  Python executable is C:\hostedtoolcache\windows\Python\3.11.9\x64\python.exe
  System Python files loaded from C:\hostedtoolcache\windows\Python\3.11.9\x64
  --------------------------------------------------------------------
  Python Pillow modules loaded from C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\site-packages\PIL
  Binary Pillow modules loaded from C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\site-packages\PIL
  --------------------------------------------------------------------
  --- PIL CORE support ok, compiled for 11.1.0.dev0
  --- TKINTER support ok, loaded 8.6
  --- FREETYPE2 support ok, loaded 2.13.3
  --- LITTLECMS2 support ok, loaded 2.16
  --- WEBP support ok, loaded 1.4.0
  --- JPEG support ok, compiled for libjpeg-turbo 3.0.4
  --- OPENJPEG (JPEG2000) support ok, loaded 2.5.2
  --- ZLIB (PNG/ZIP) support ok, loaded 1.3.1.zlib-ng, compiled for zlib-ng 2.2.2
  --- LIBTIFF support ok, loaded 4.6.0
  --- RAQM (Bidirectional Text) support ok, loaded 0.10.1, fribidi 1.0.16, harfbuzz 10.0.1
  --- LIBIMAGEQUANT (Quantization method) support ok, loaded 2.16.0
  *** XCB (X protocol) support not installed
  --------------------------------------------------------------------

I took inspiration from https://github.com/python-pillow/Pillow/pull/8495#issuecomment-2436585125 to make a benchmark with multiple compression values. It seems that zlib-ng usually produces a slightly larger file but in much less time during compression.

Script and results (click to expand)

```python import io import os import timeit from PIL import Image PATH = "Tests/images/hopper.png" # or "Tests/images/effect_spread.png" REPEAT = 1000 print(f"{Image.__version__ = }") print(f"{Image.core.zlib_version = }") print(f"Testing {PATH = } with {REPEAT} repetitions") with open(PATH, "rb") as f: data_in = f.read() reader = io.BytesIO(data_in) writer = io.BytesIO() img = Image.open(reader) img.load() def read_png(): reader.seek(0, os.SEEK_SET) return Image.open(reader) def write_png(compress_level): writer.seek(0, os.SEEK_SET) writer.truncate(0) img.save(writer, "PNG", compress_level=compress_level) t = timeit.timeit(read_png, number=REPEAT * 10) print(f"read PNG: time = {t:f} (sec)") for compress_level in range(0, 10): t = timeit.timeit(lambda: write_png(compress_level), number=REPEAT) print( f"write PNG: time = {t:f} (sec); size = {len(writer.getvalue())} bytes; {compress_level = }" ) Image.open(io.BytesIO(writer.getvalue())).show() # Image.__version__ = '11.1.0.dev0' # Image.core.zlib_version = '1.3.1' # Testing PATH = 'Tests/images/hopper.png' with 1000 repetitions # read PNG: time = 0.157310 (sec) # write PNG: time = 0.376219 (sec); size = 49353 bytes; compress_level = 0 # write PNG: time = 0.945004 (sec); size = 32403 bytes; compress_level = 1 # write PNG: time = 1.046024 (sec); size = 32156 bytes; compress_level = 2 # write PNG: time = 1.238888 (sec); size = 31792 bytes; compress_level = 3 # write PNG: time = 1.549707 (sec); size = 30715 bytes; compress_level = 4 # write PNG: time = 1.881609 (sec); size = 30529 bytes; compress_level = 5 # write PNG: time = 2.527365 (sec); size = 30343 bytes; compress_level = 6 # write PNG: time = 3.075476 (sec); size = 30277 bytes; compress_level = 7 # write PNG: time = 5.036960 (sec); size = 30183 bytes; compress_level = 8 # write PNG: time = 6.598194 (sec); size = 30166 bytes; compress_level = 9 # Image.__version__ = '11.1.0.dev0' # Image.core.zlib_version = '1.3.1.zlib-ng' # Testing PATH = 'Tests/images/hopper.png' with 1000 repetitions # read PNG: time = 0.178209 (sec) # write PNG: time = 0.382191 (sec); size = 49353 bytes; compress_level = 0 # write PNG: time = 0.549274 (sec); size = 40354 bytes; compress_level = 1 # write PNG: time = 0.770737 (sec); size = 31458 bytes; compress_level = 2 # write PNG: time = 0.934707 (sec); size = 31096 bytes; compress_level = 3 # write PNG: time = 0.977891 (sec); size = 30836 bytes; compress_level = 4 # write PNG: time = 1.101530 (sec); size = 30771 bytes; compress_level = 5 # write PNG: time = 1.212539 (sec); size = 30649 bytes; compress_level = 6 # write PNG: time = 1.383462 (sec); size = 30217 bytes; compress_level = 7 # write PNG: time = 1.946067 (sec); size = 30175 bytes; compress_level = 8 # write PNG: time = 2.788035 (sec); size = 30166 bytes; compress_level = 9 # Image.__version__ = '11.1.0.dev0' # Image.core.zlib_version = '1.3.1' # Testing PATH = 'Tests/images/effect_spread.png' with 1000 repetitions # read PNG: time = 0.096424 (sec) # write PNG: time = 0.410247 (sec); size = 49353 bytes; compress_level = 0 # write PNG: time = 1.194126 (sec); size = 42199 bytes; compress_level = 1 # write PNG: time = 1.164452 (sec); size = 42096 bytes; compress_level = 2 # write PNG: time = 1.182063 (sec); size = 41977 bytes; compress_level = 3 # write PNG: time = 1.404397 (sec); size = 41011 bytes; compress_level = 4 # write PNG: time = 1.531614 (sec); size = 40969 bytes; compress_level = 5 # write PNG: time = 1.720916 (sec); size = 40903 bytes; compress_level = 6 # write PNG: time = 1.806385 (sec); size = 40896 bytes; compress_level = 7 # write PNG: time = 1.914409 (sec); size = 40889 bytes; compress_level = 8 # write PNG: time = 1.936970 (sec); size = 40889 bytes; compress_level = 9 # Image.__version__ = '11.1.0.dev0' # Image.core.zlib_version = '1.3.1.zlib-ng' # Testing PATH = 'Tests/images/effect_spread.png' with 1000 repetitions # read PNG: time = 0.088179 (sec) # write PNG: time = 0.412940 (sec); size = 49353 bytes; compress_level = 0 # write PNG: time = 0.607201 (sec); size = 48241 bytes; compress_level = 1 # write PNG: time = 0.852157 (sec); size = 41287 bytes; compress_level = 2 # write PNG: time = 1.026876 (sec); size = 41168 bytes; compress_level = 3 # write PNG: time = 1.044624 (sec); size = 41134 bytes; compress_level = 4 # write PNG: time = 1.200349 (sec); size = 41120 bytes; compress_level = 5 # write PNG: time = 1.214951 (sec); size = 41118 bytes; compress_level = 6 # write PNG: time = 0.983906 (sec); size = 40895 bytes; compress_level = 7 # write PNG: time = 0.989961 (sec); size = 40889 bytes; compress_level = 8 # write PNG: time = 1.540352 (sec); size = 40889 bytes; compress_level = 9 ```

Looking at the build logs, zlib-ng seems to have been properly detected by webp, libtiff, libpng, and freetype.

TODO:

• [x] Run the test suite on Windows arm64. I can use a borrowed M2 MacBook, but I won't have access to it at least until Monday.
• [x] We should also update the Linux...
• [ ] ...and macOS builds for consistency if we decide that we do want to go with zlib-ng.

[dofuuz]

zlib-ng v2.2.2 x86_64-compat was not released because of CI test error. It'll be released at next version again. So, downloading pre-built binary is still a valid way.

Anyway, building with CMake is also a fine solution. I'll follow Pillow devs' decision and close my PR #8495 when it's decided.

[hugovk]

Here's a chart of the write times for different compression levels and images:

• hopper.png: 1.3.1 zlib-ng (red) is much better than 1.3.1 (blue)
• effect_spread.png: 1.3.1 zlib-ng (green) is noticeably better than 1.3.1 (yellow)

Summary: zlib-ng is much faster.

---

And file sizes:

• compression level 0: same for all
• compression level 1: 1.3.1 zlib-ng is much worse than 1.3.1
• compression level 2-3: 1.3.1 zlib-ng is slightly better than 1.3.1
• compression level 4-6: 1.3.1 zlib-ng is even more slightly worse than 1.3.1
• compression level 7-9: about the same, zlib-ng ever so slightly better than 1.3.1

Summary: zlib-ng has same file sizes for compression level 0 and bigger files for compression level 1, but if you care about size, any higher compression level gives more or less the same file size (and zlib-ng is faster).

[nulano]

I've now checked that the Windows arm64 wheels pass the test suite on an MacBook Pro M2 Max (tested Python versions: 3.9.10, 3.10.11, 3.11.6, 3.12.0, 3.13.0).

Benchmark results comparing the Python 3.13 wheels of the 11.0.0 release and the pull request build (click to expand)

``` Image.__version__ = '11.0.0' Image.core.zlib_version = '1.3.1' Testing PATH = '../Pillow-11.0.0/Pillow-11.0.0/Tests/images/hopper.png' with 1000 repetitions read PNG: time = 0.589200 (sec) write PNG: time = 0.555782 (sec); size = 49353 bytes; compress_level = 0 write PNG: time = 1.114117 (sec); size = 32403 bytes; compress_level = 1 write PNG: time = 1.198036 (sec); size = 32156 bytes; compress_level = 2 write PNG: time = 1.402062 (sec); size = 31792 bytes; compress_level = 3 write PNG: time = 1.758227 (sec); size = 30715 bytes; compress_level = 4 write PNG: time = 2.065863 (sec); size = 30529 bytes; compress_level = 5 write PNG: time = 2.774447 (sec); size = 30343 bytes; compress_level = 6 write PNG: time = 3.542967 (sec); size = 30277 bytes; compress_level = 7 write PNG: time = 5.532757 (sec); size = 30183 bytes; compress_level = 8 write PNG: time = 7.352626 (sec); size = 30166 bytes; compress_level = 9 Image.__version__ = '11.1.0.dev0' Image.core.zlib_version = '1.3.1.zlib-ng' Testing PATH = '../Pillow-11.0.0/Pillow-11.0.0/Tests/images/hopper.png' with 1000 repetitions read PNG: time = 0.569872 (sec) write PNG: time = 0.529697 (sec); size = 49353 bytes; compress_level = 0 write PNG: time = 0.723248 (sec); size = 40354 bytes; compress_level = 1 write PNG: time = 0.942190 (sec); size = 31458 bytes; compress_level = 2 write PNG: time = 1.009014 (sec); size = 31096 bytes; compress_level = 3 write PNG: time = 1.053823 (sec); size = 30836 bytes; compress_level = 4 write PNG: time = 1.071575 (sec); size = 30771 bytes; compress_level = 5 write PNG: time = 1.170958 (sec); size = 30649 bytes; compress_level = 6 write PNG: time = 1.503532 (sec); size = 30217 bytes; compress_level = 7 write PNG: time = 2.005649 (sec); size = 30175 bytes; compress_level = 8 write PNG: time = 2.626763 (sec); size = 30166 bytes; compress_level = 9 Image.__version__ = '11.0.0' Image.core.zlib_version = '1.3.1' Testing PATH = '../Pillow-11.0.0/Pillow-11.0.0/Tests/images/effect_spread.png' with 1000 repetitions read PNG: time = 0.234989 (sec) write PNG: time = 0.550197 (sec); size = 49353 bytes; compress_level = 0 write PNG: time = 1.305417 (sec); size = 42199 bytes; compress_level = 1 write PNG: time = 1.334536 (sec); size = 42096 bytes; compress_level = 2 write PNG: time = 1.384041 (sec); size = 41977 bytes; compress_level = 3 write PNG: time = 1.622062 (sec); size = 41011 bytes; compress_level = 4 write PNG: time = 1.721847 (sec); size = 40969 bytes; compress_level = 5 write PNG: time = 1.927218 (sec); size = 40903 bytes; compress_level = 6 write PNG: time = 2.054222 (sec); size = 40896 bytes; compress_level = 7 write PNG: time = 2.198446 (sec); size = 40889 bytes; compress_level = 8 write PNG: time = 2.199249 (sec); size = 40889 bytes; compress_level = 9 Image.__version__ = '11.1.0.dev0' Image.core.zlib_version = '1.3.1.zlib-ng' Testing PATH = '../Pillow-11.0.0/Pillow-11.0.0/Tests/images/effect_spread.png' with 1000 repetitions read PNG: time = 0.240964 (sec) write PNG: time = 0.538218 (sec); size = 49353 bytes; compress_level = 0 write PNG: time = 0.799972 (sec); size = 48241 bytes; compress_level = 1 write PNG: time = 1.091412 (sec); size = 41287 bytes; compress_level = 2 write PNG: time = 1.177194 (sec); size = 41168 bytes; compress_level = 3 write PNG: time = 1.191972 (sec); size = 41134 bytes; compress_level = 4 write PNG: time = 1.195040 (sec); size = 41120 bytes; compress_level = 5 write PNG: time = 1.190594 (sec); size = 41118 bytes; compress_level = 6 write PNG: time = 1.280668 (sec); size = 40895 bytes; compress_level = 7 write PNG: time = 1.293338 (sec); size = 40889 bytes; compress_level = 8 write PNG: time = 1.738161 (sec); size = 40889 bytes; compress_level = 9 ```

[hugovk]

Similar trend for time and identical numbers for size: