protomaps / go-pmtiles

Single-file executable tool for working with PMTiles archives
BSD 3-Clause "New" or "Revised" License
356 stars 49 forks source link

add minzoom option to extract command #120

Closed JesseCrocker closed 8 months ago

JesseCrocker commented 8 months ago

There is already a maxzoom option for the extract command, so a minzoom option seemed like a natural compliment to it.

bdon commented 8 months ago

Thanks for the PR! Copying some experimental results here:

go run main.go extract https://build.protomaps.com/20240116.pmtiles out.pmtiles --minzoom=10 --maxzoom=13  --dry-run

fetching 339 dirs, 339 chunks, 1 requests
Region tiles 89128960, result tile entries 18116946
fetching 18116946 tiles, 26798 chunks, 26722 requests

go run main.go extract https://build.protomaps.com/20240116.pmtiles out.pmtiles --minzoom=9 --maxzoom=13  --dry-run

fetching 343 dirs, 343 chunks, 1 requests
Region tiles 89391104, result tile entries 18343948
fetching 18343948 tiles, 4 chunks, 1 requests

The PMTiles design is optimized for extracting a sub pyramid with all overviews (minzoom=0) in a very small# of requests. extracting with minzoom like in the above example can result in tens of thousands of requests depending on tile de-duplication. The above is roughly the worst-case example (whole planet OSM basemap).

If extracting minzoom like this is a common part of your workflow, you could adjust the way tiles are de-duplicated; you could salt the content hash for de-duplication using the Z of the tile. That would mean minzoom-maxzoom pyramids are always a small # of requests, at the cost of duplicating those tiles at every zoom level.

These are things we ought to document in https://docs.protomaps.com/pmtiles/cli