Proposal: Better support for overzooming via very large tile extents

[ChrisLoer]

MapLibre currently "supports" up to 10-levels of overzooming (e.g. SourceCache.maxOverzooming = 10;), but its actual behavior at that level of overzooming is not very good -- it essentially makes gigantic 524,288px square tile and then renders a small portion of it on the viewport. There are I think two main problems with this:

• The WebGL/shader-enforced tile extent of 8192 (or 16384 after https://github.com/maplibre/maplibre-gl-js/pull/1244) means that each tile unit covers up to 64 pixels -- that makes for a very imprecise map!
• Elements line line labels that repeat based on the overzoomed tile scale end up with an enormous number of copies to fill the gigantic tile. This is bad for performance, but also once more than 64k worth of glyphs are generated for a tile, you'll start seeing garbled labels and potentially even throw exceptions during rendering.

High overzooming is unavoidable in non-giant tilesets

The vector tile spec supports extents going all the way up to max_uint32, but why is it useful to have such large extents? An extent of 8192 works fine for tilesets that go out to z15 or z16. The problem is that making tile sets that go all the way out to z15 is expensive (more than a billion tiles!). Making those giant tile sets is just part of the job for global basemaps like Mapbox Streets or MapTiler Planet, but when you're working on data visualizations, there are lots of datasets that are: (1) big enough to need tiling, and (2) small enough that you should expect to generate a tileset in minutes without needing a compute cluster.

Using a large tile extent allows you to tile your dataset only as far as is necessary to keep maxzoom tile data size within your target limits, and then by increasing the extent of your maxzoom tiles, you can avoid giving up any precision. This is what Felt does with Tippecanoe when we tile user uploads to our pipeline -- unfortunately, when we moved from Protomaps.js to MapLibre, we lost the precision because the extent gets converted down to 8192 for display. Moving to MapLibre was great, but this is the biggest remaining regression for us.

Proposed solution: reuse GeoJSON-VT

Luckily, MapLibre already pretty much knows how to do good overzooming for high precision tiles, because it does it for GeoJSON sources, which are essentially just a single-tile source with very high precision. The geojson-vt library takes a single high-precision tile (the GeoJSON) as an input, and then answers requests for Vector Tiles from MapLibre by chopping up the source tile into smaller geometries on the fly. Points and lines are easy to do (just take the subset within the new tile bounds), but polygons are a bit trickier because you have to make new polygons that fit within the new tile. geojson-vt makes this fast in part by maintaining an in-memory tile-pyramid so that each new tile can take advantage of any chopping work that already happened above it in the pyramid.

My proposal is to adapt geojson-vt so that instead of just working with GeoJSON, it can be used for client-side overzooming of any high detail tile. Imagine a tileset that's tile to z8, with the tiles at z8 having an extent of 65536:

• From z1 to z8, MapLibre just shows the tiles using the existing logic (the extra precision in the extent is thrown away at z8, but it doesn't matter at this zoom level).
• At z9, when MapLibre requests a tile from the source, it finds the nearest parent (in this case, the z8 tile), loads it, and then uses geojson-vt to generate a z9 tile.
• At zN, it can take advantage of the zN-1 tiles that are already in memory to generate a zN tile
• As long as any tiles that are children of the original z8 tile are in the map's tile pyramid, keep the z8 tile in as well.
• Once the z8 tile leaves the map's pyramid, you also clear the in-memory geojson-vt pyramid.

I prototyped this locally (without doing the pyramid cleanup parts), and it seemed to work pretty well -- took about a day to get working. I did have to reach into some geojson-vt internals a little bit, so we would probably want to put a PR into geojson-vt or fork it in order to enable this.

Variants

There are a few different ways I can imagine this plugging into MapLibre

Add this behavior directly to Vector Tile Source

My preferred solution here would be to just build this into the existing Vector Tile Source logic. Nothing about the existing style spec would have to change, the only difference would be that at source maxzoom, the Vector Tile Source would watch for high-extent tiles, and if it found them it would deduce "I have enough precision to provide overzooming for X more zoom levels" -- it would tell the SourceCache (so that the "covering tiles" logic knew to keep requesting tiles), and then provide the overzoomed tiles using the geojson-vt behavior.

I like this approach because:

• No change to style spec/docs/etc.
• Minimal extra bundle size (most of the code needed is already included in geojson-vt)
• No change to behavior for sources that aren't using high extents (which is probably almost all of them, since this hasn't worked well previously!)
• The change to behavior for sources with high extents should be a pure improvement

Add a new Source type derived from Vector Tile Source

My prototype implemented this behavior as a "HighDetailSource" that extended "VectorTileSource". The source was defined with a max "network" zoom as well as a regular maxzoom (where the "network" zoom determined when it decided to start making tiles locally instead of asking the network). Doing it this way avoids having to touch the SourceCache logic at all or do any reasoning about what the maxzoom should be based on the extent -- but it makes the user specify two zoom levels, and adds a whole new source type to the style spec, so it feels a bit clunky to me.

Make Sources more "pluggable"

This is kind of a meta-solution, but if we think this functionality is too "niche" to be a core part of MapLibre, you could imagine supporting extra source types as a "plugin". I haven't really sketched out what this would look like, but I think at the very least you want access to all vector-tile->gl-bucket logic that's in VectorTileSource. This is already half-exposed with the LoadVectorData callback, but to make the "high detail source" work you would have to expose at least a few more things (like hooking unloadTile so you'd know when to do cleanup).

Rationale / Impact

I think the main workarounds here are:

• Use GeoJSON (but this becomes less feasible as datasources get larger)
• Generate your tilesets to higher zoom if you want more precision (but this is expensive)
• Build overzooming support into your tileserver instead of into the client (this is an option we're looking at in Felt, but I think this is a higher impact improvement if we can do it in maplibre instead of every different user of maplibre having to figure out how to do it in their respective tile servers)

And of course, the status quo has worked for years despite its shortcomings! I know Felt has a use for this, and I think it's "medium size vector tilesets" are a very real use case for lots of data visualization, but I can't really make the case for what other groups would use this off the bat.

cc @wipfli @HarelM @ibesora @bdon @dnomadb

[wipfli]

Thanks for the proposal @ChrisLoer. I think this would indeed be an improvement and would vote for option one, "Add this behavior directly to Vector Tile Source".

Can we do the same in MapLibre Native?

[HarelM]

Would be great to have this in. I would recommend starting with a PR to geojson-vt to see if someone there would like to incorporate it as well. If we decide to fork geojson vt I would also like to take into consideration the work we started to update a geojson source, which currently is not completed in the sense that the updates are rebuilding the index instead of updating it, so we keep the data twice in memory. But this is an unrelated note.

[ChrisLoer]

Can we do the same in MapLibre Native?

🤞 I think it would be a relatively straightforward port, because geojson-vt-cpp is a pretty direct port of geojson-vt. I don't think Felt would be able to "pay for" the work though.

I would recommend starting with a PR to geojson-vt to see if someone there would like to incorporate it as well.

Yeah that would be best. I need to dig into this more to see how minor a change we can get away with.

[bdon]

+1 to Option 1, Add this behavior directly to Vector Tile Source

unfortunately, when we moved from Protomaps.js to MapLibre, we lost the precision because the extent gets converted down to 8192 for display. Moving to MapLibre was great, but this is the biggest remaining regression for us.

Just for the record, protomaps.js has the same ailment related to line label copies - it doesn't have precision loss though because the overzoomed tile can have an arbitrarily large extent instead of being limited to 8192

Build overzooming support into your tileserver instead of into the client

I think the latency drawback here is enough to dismiss it as a first-class solution? It might be useful for server-side vector tile compositing from different maxzooms, though.

[ChrisLoer]

I think the latency drawback here is enough to dismiss it as a first-class solution? It might be useful for server-side vector tile compositing from different maxzooms, though.

Maybe I'm not thinking about this in the right frame, but isn't the latency question just a matter of computation that has to happen on one side or the other? e.g. tile might be served slightly faster without server-side overzooming, but then require slightly more time to get ready before display on the client side?

[bdon]

@ChrisLoer I meant comparing an overzoomed tile client-rendered (no network request) vs server-side (let's say 100ms for a cache hit, also likely incurs a billable event)

Are there adverse situations where a client-overzoomed tile in Option 1 takes more than a couple hundred milliseconds?

[ChrisLoer]

Just for context -- we ended up implementing server-side overzooming at Felt as a matter of expediency, so our motivation to push this through is not so strong now (although I still think it would be a useful general feature for maplibre to have).