systemed / tilemaker

Make OpenStreetMap vector tiles without the stack
https://tilemaker.org/
Other
1.44k stars 229 forks source link

warn the user when using sub-optimal PBFs #601

Closed cldellow closed 9 months ago

cldellow commented 9 months ago

Tilemaker's documentation suggests that users can get PBFs from BBBike: https://github.com/systemed/tilemaker/blob/1da4be97dc4ac7f11daf3f417c7ca0a6a34ae47f/docs/VECTOR_TILES.md?plain=1#L25

BBBike uses osmconvert to generate PBFs. Reading through the osmconvert code 1, osmconvert will pack up to 31 MB of data into each block.

PBFs from Geofabrik use Osmium, which defaults to packing 8,000 objects into each block, resulting in blocks that are more like ~60 KB in size.

More, smaller blocks are better for Tilemaker:

I emailed BBBike's maintainer. He wants to one day move to Osmium, and is understandably not keen on patching osmconvert in the interim.

He did point out that a user can just run osmium cat on a PBF from BBBike to rejig its innards.

This PR detects when a PBF is suboptimal, warns the user, and provides an explanation of how to fix it.

For one of my BBBike PBFs, this results in processing time dropping from 90 seconds to 28 seconds. (osmium cat itself only takes 17 seconds, and in any case, only has to be run a single time.)

systemed commented 9 months ago

Good spot - I pretty much exclusively use Geofabrik PBFs so had never noticed this!