felt / tippecanoe

Build vector tilesets from large collections of GeoJSON features.
BSD 2-Clause "Simplified" License
982 stars 84 forks source link

Suggestion: `--drop-by-attribute-as-needed=ATTRIB_NAME` #132

Open amandasaurus opened 1 year ago

amandasaurus commented 1 year ago

Suggestion: add a new flag to drop objects from a MVT tile (if needed to keep within the tile size), and drop e.g.: If the tile is too big, “sort” a objects by an attribute, and keep the top N objects which will fit inside this size (and including the N+1 object (when ordered by this attribute) would cause the tile to exceed the limit).

The real world example is that I have millions of little line segements, representing lots of waterways (rivers, streams etc). Each segment has a float for the size of the total river system it's part of. To make MVT pmtiles, I need to drop objects. The logical way is to keep the segments for large river basins.

The code is osm-river-basins on Github, and my tippecanoe command is here in functions.sh. I tried my best, and people are still noticing that it produces gaps in the lines. I believe, a --drop-as-needed-by-attribute=ATTRIB would solve this issue.

Obv. some reverse sorting option would probably be useful for others too.

dabreegster commented 1 year ago

I would also love to see this, for https://github.com/dabreegster/routing-engines. I've got lines covering every street segment in region to country-sized areas, with a count attribute representing betweenness centrality or some measure of importance. At low zooms with --drop-fraction-as-needed --extend-zooms-if-still-dropping, it looks like this: Screenshot from 2023-10-06 10-50-15 I've started experimenting with manually specifying a minimum count for low zooms, like -z13 -j '{"*": [">", "count", 100]}', but I'm not sure yet how to determine good thresholds for a given dataset. Ideally I'd keep as many lines per tile as possible up to the size / feature limits, but if I have to drop something, sort by this count attribute within that tile.

e-n-f commented 1 year ago

Good idea! I will have to look into adding something like this.

stalker314314 commented 2 months ago

Same as @dabreegster - I also cover all streets (https://maxspeeds.openstreetmap.rs/), but on lower zooms, it would be great to sort them and keep first motorways, then trunks, then primaries... now it is either keeping random elements (and having gaps on major routes), or I can filter with -C and use tile-join, but then there is room for more features for some tiles.

(additional not-so-important fact - I thought that --preserve-input-order or anything else in that section https://github.com/mapbox/tippecanoe?tab=readme-ov-file#reordering-features-within-each-tile will help me with this, but took me some time to figure it is dead end - wrong terminology for my use case:D)

dabreegster commented 2 months ago

I started https://github.com/Urban-Analytics-Technology-Platform/lines2pmtiles/ as an experiment to generate pmtiles just focusing on linestrings, with this attribute-based sorting upfront. But it's far from being usable or having competitive performance. It'd be amazing to have support in tippecanoe for this.

Robinlovelace commented 1 week ago

I think a pre-processing step that filters out less important features for lower zoom levels is the way forward. Not sure how to combine the 'use this dataset for high zooms and this one for low zooms' elements into a single .pmtiles output but believe it's possible, will take a look at the docs.