search

mapbox / tippecanoe

Build vector tilesets from large collections of GeoJSON features.
BSD 2-Clause "Simplified" License
2.72k stars 432 forks source link

Change DEFLATE from zlib to libdeflate #674

Open Plantain opened 5 years ago

Plantain commented 5 years ago

zlib is no longer a good choice for a fast and performant DEFLATE algorithm. Just having picked one other that I found, https://github.com/ebiggers/libdeflate, I found it produces ~20% smaller files in less time.

For example, on a large tile of GeoJSON points: 2116919B Uncompressed 375282B libdeflate -12 378332B libdeflate -9 449989B tippecanoe/zlib -9

For a small tile with a few polygons, the difference was less dramatic but still positive: 53477B Uncompressed 40501B libdeflate -12 40552B libdeflate -9 41121B tippecanoe/zlib -9

I see another mapbox project is in the process of migrating the same library - https://github.com/mapbox/gzip-hpp/pull/25

I have not compared to any other zlib libraries.

e-n-f commented 5 years ago

Thanks for the suggestion. As you saw in the gzip-hpp ticket, we have already started looking into this, and libdeflate does seem to do a better job in most cases.