Untangle composited tiles + make composite requests on the client

[kkaefer]

Mapbox Studio allows creating "composited" sources. This means it'll use mapbox:// tile URLs that instruct Mapbox servers to combine multiple vector tiles into one, e.g. we're frequently using mapbox://mapbox.mapbox-streets-v8,mapbox.mapbox-terrain-v2, which contains all data layers from both tilesets. We do this to reduce the number of HTTP request necessary to obtain data, and we required this functionality to work around the lack of support for cross-source label placement (which recently got fixed).

Mapbox GL is entirely unaware of these composited URLs and just treats them as their own tileset. This means that when using multiple styles that contain source URLs that contain a different composite tile subset or no compositing, we won't be able to reuse the data we already have in the cache. E.g. a mapbox://mapbox.streets-v8 source is unable to reuse any cached data from a mapbox://mapbox.mapbox-streets-v8,mapbox.mapbox-terrain-v2 tileset.

To improve this situation, we could add source compositing to GL Core:

• When storing composited tiles into the cache database, we separate the layers into individual entries by layer. For this to work, we need metadata about the layer association. We could either reuse the data stored in the TileJSON file, or add a field to the layer itself that stores the name of the tileset it belongs to ("namespace").
• Remove source compositing from Studio so we never store composited URLs in the stylesheet file itself.
• When requesting a tile in GL Core, we'd have to wait until the end of the loop iteration in order to collect mapbox:// requests to the same tile ID and replace those requests with a composited one. We could either build this into the DefaultFileSource hierarchy (OfflineDatabase + OnlineFileSource), or directly into the tile requesting mechanism.
• Add a way to disable source compositing for certain sources that have different caching needs (e.g. traffic) in order to prevent expiring longer-lived content too frequently.
• Add support for styles that still use composited URLs by doing separate cache requests + compositing them in OfflineDatabase.

/cc @mapbox/studio @jfirebaugh @brunoabinader

[brunoabinader]

When storing composited tiles into the cache database, we separate the layers into individual entries by layer.

If I understood it correctly, this would involve 1) reading the Mapbox Vector Tile data from each received tile, and 2) serialize each mapbox::vector_tile::layer in individual entries.

For this to work, we need metadata about the layer association. We could either reuse the data stored in the TileJSON file, or add a field to the layer itself that stores the name of the tileset it belongs to ("namespace").

The former option is possible via the vector_layers TileJSON extension (pending documentation). From that, we can extract each individual source vector layer's ID and its respective source ID. This information can be contained inside the Tileset class, and further accessed to store/compare cached tile entries.

On the database, we would then be adding two new columns: source_id and source_layer_id. The source ID can be obtained either from vector_layers (if composite/present) or from the source itself.

When selecting cached tile entries, we'd then no longer use url_template as key, but source_id and source_layer_id instead. @kkaefer is this what you have in mind?

[kkaefer]

When selecting cached tile entries, we'd then no longer use url_template as key, but source_id and source_layer_id instead. @kkaefer is this what you have in mind?

Changing the schema is separately ticketed in https://github.com/mapbox/mapbox-gl-native/issues/10866. We'd still use the URL template in order to support non-Mapbox tilesets (since they don't have the notion of a source_id, but instead of using the composited URL, we'd only use uncomposited URLs.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically detected as stale because it has not had recent activity and will be archived. Thank you for your contributions.