search

mapbox / rio-rgbify

Encoded arbitrary bit depth rasters in pseudo base-256
MIT License
99 stars 36 forks source link

Support writing PNGs with filter 3 #3

Open kkaefer opened 8 years ago

kkaefer commented 8 years ago

Using filter 3 results in ~30% smaller PNG files in some situations.

dnomadb commented 8 years ago

@kkaefer It appears that GDAL http://www.gdal.org/frmt_various.html#PNG nor Pillow http://pillow.readthedocs.io/en/3.1.x/handbook/image-file-formats.html#png support that option; I added a webp output option that results in files ~1/2 the size, but at 2x+ the creation time.

vincentsarago commented 6 years ago

I added a webp output option that results in files ~1/2 the size, but at 2x+ the creation time.

I think it's because we use lossless=True, we could maybe use another config without impacting the quality too much. e.g in mapnik we use default encoding is quality=75 and lossless=0. and in rio-tiler we use quality=80

dnomadb commented 6 years ago

because we use lossless=True

Yeah, which we have to, as any lossy compression or even resampling will corrupt the encoding.

andrewharvey commented 6 years ago

I wanted to chime in here from a users perspective higher up the Mapbox stack.

The first thing I noticed back when I tested out GL JS's hillshading was using raster-dem/terrain-rgb tiles were much slower to load than the original vector hillshading layers from the mapbox terrain tileset.

On a sample page the terrain-rgb tiles took up 3.2M of 4.1M in total, with the normal vector tiles poping in at about 1.5s, labels at about 2s, but then it takes until 6s for the terrain tiles to finish downloading. We should be able to do better than that.

So I took a look and found asking for terrain-rgb webp tiles only serves pngs.

https://a.tiles.mapbox.com/v4/mapbox.terrain-rgb/9/85/197@2x.webp
> identify 197@2x.webp
PNG 512x512 512x512+0+0 8-bit sRGB

I tried a few compression options to see what affect that would have:

> for f in tiles/*; do b=`basename $f`; pngcrush -brute $f pngcrush/$b; done
> for f in tiles/*; do b=`basename $f`; cwebp -lossless -q 100 -m 6 $f -o cwebp/$b; done
method total size
original 3.3M
pngcrush 2.4M ~30% improvement
cwebp 1.8M ~50% improvement

That matches exactly the findings of @kkaefer and @dnomadb given pngcrush I believe is finding the optimal png filter, both of them doing everything lossless.

Given webp isn't well supported that 30% gain on PNGs by changing the filter would be a huge win for users, together with the availability of webp terrain-rgb tiles for a 50% gain.

andrewharvey commented 3 years ago

Given webp isn't well supported that 30% gain on PNGs by changing the filter would be a huge win for users, together with the availability of webp terrain-rgb tiles for a 50% gain.

Bit off topic but I was happy to read this has been done now in the new Mapbox Raster Data API announced at https://www.mapbox.com/blog/mapbox-gl-js-v2-3d-maps-camera-api-sky-api-launch