Style spec: provide a mechanism for composing styles

[anandthakker]

tl;dr: The style spec needs a mechanism that allows for 'composing' styles / style layers.

A common use case for GL JS and GL Native is dynamically annotating an otherwise static "basemap" style with markers, routelines, areas-of-interest, etc. This poses difficulties at present, because the dynamic layers usually need to be placed somewhere in the middle of the "basemap" layers, which means:

1. Knowing where in the stack to place the layer depends on detailed knowledge of the basemap style's implementation. Concretely, this often means inspecting Mapbox-provided styles like Mapbox Streets or Outdoors to find the right layer id before which to insert new layers, and hardcoding that layer id into a site/app's implementation.
2. Manipulating style layers in an unweildy manner. Especially if the dynamic content involves adding/updating more than just one or two style layers, using the basic addLayer(), set{PaintLayout}Property() APIs can lead to spaghetti-like 🍝 code.

"Smart setStyle" could provide some relief on number 2, since it allows authors to just write a function that produces the style they want. However, without a mechanism for composing styles, it only moves the problem, it doesn't solve it: a 'reactive' style-building function would still have to do ad-hoc surgery on the basemap to produce the desired style.

Existing proposals:

• "placeholder areas" or "marks": add a way to identify a certain stack of layers or a certain location in the list of layers within a style, so that those stacks / locations can be referred to in, e.g., addLayer() (and maybe things like toggling visibility).
• "nested styles": add a style layer type that recursively includes the layers from a different style, so that basemap styles could be shipped in multiple pieces (mapbox-streets-v9-bottom, mapbox-streets-v9-top) that authors could combine with their own layers.

[stevage]

Nice problem statement. I run into variations onto this theme a lot. A couple of awkward issues:

• waiting for the style to fetch before you can construct new style layers to add to it (see #3964)
• having to choose between manipulating the whole style JSON file, or, once it's loaded, having to go through the style API (addLayer()) etc.

I guess I have issue 1 (knowing where to add my dynamic styles) but I never really noticed or bothered trying to solve it. I just stick mine at the end, over the top of other layers.

Maybe a convenient interface would look something like:

map = new mapboxgl.Map({ .... });
map.addStyle({
   name: 'circles',
   before: 'place-labels',
   style: {
     ...
   }
);

Would it make sense to pre-define a couple of standard layer breakpoints that all maps are expected to conform to? Maybe that's too crazy. :)

[anandthakker]

cc @mapbox/gl

[1ec5]

Would it make sense to pre-define a couple of standard layer breakpoints that all maps are expected to conform to?

Standard “bookmark” layers would have to be limited to cross-platform concepts, so they wouldn’t for instance be usable for a user dot/puck, route lines, and GL annotations, features that would benefit from a bookmark and are supported in the mobile SDKs but not GL JS.

[lucaswoj]

"Nested" styles helps solve the common preserve-my-custom-layers-while-switching-"basemaps" problem. It may also help the carto team reuse some design elements across map styles (like label layers).

[lucaswoj]

One more proposal:

We encourage developers to compose styles using the full power of their native language and publish tools which facilitate this workflow:

map.setStyle(createStyle({
    traffic: true,
    satellite: true
}));

This dovetails nicely with #3621 and #4000.

[1ec5]

The general concept behind https://github.com/mapbox/mapbox-gl-js/issues/4225#issuecomment-280506023 would also help us achieve feature parity with competing SDKs on the native platforms: mapbox/mapbox-gl-native#8071.

[Spaxe]

(Thanks @andrewharvey for pointing me here.)

Here is an example where I needed to nest custom traffic data between town labels, but on top of roads:

I imagine there are heaps more different use cases where inserting custom layers are necessary, but between different layers. Per #4690, I incorrectly used dataset ids and not layer ids, even though the style details show dataset ids in bold, and the tooltip says "layer".

On this stackexchange post, the order of map items are well defined, and I am sure many people would agree that this is largely correct in order of rendering:

layers:
    - landuse
    - waterway
    - water
    - aeroway
    - data
    - barrier_line
    - building
    - landuse_overlay
    - tunnel
    - road
    - bridge
    - admin
    - country_label_line
    - country_label
    - marine_label
    - state_label
    - place_label
    - water_label
    - poi_label
    - road_label
    - waterway_label
    - housenum_label

However, Mapbox allows you to import individual layers from datasets, so these names can't be used without manually adding in metadata. Using the dataset names directly (like "place_label") also doesn't work, because their individual layers may be arranged separately.

That said, I support @stevage's suggestion for 'breakpoints' that are very common in use. For instance, in this ascending order:

• basemap
• landuse
• water
• admin
• roads
• labels

Most data will fit onto roads, such as Traffic and Navigation routes, which go on top of roads and transport routes. Geographical datasets that don't concern roads will fit onto water, such as Choropleths and Urban Areas, which need to be displayed under administration / political boundaries.

In order to avoid confusion with before, the 2nd argument of addLayer(), I support the syntax @stevage proposed, but with a different key:

map = new mapboxgl.Map({ .... });
map.addStyle({
   name: 'urban-walkability',
   onto: 'water',
   style: {
     ...
   }
);

This proposed onto property, when present, ignores the 2nd before argument and inserts the data before the appropriated layer. The string supplied must be either one of the proposed 'breakpoints'. For user-defined styles in Mapbox Studio, the breakpoints need to be specified in the Style Editor. Ideally, like this:

If the style does not define breakpoints, this proposed property issues a warning in the browser.

Existing styles would need to be modified to have these breakpoints defined. For users, they can drag layers in between these breakpoints. For existing styles, they could come with breakpoints. This also means, hopefully, that when you import new dataset from existing styles, it would import into the correct breakpoints.

Empty styles would come with these breakpoints predefined, although they would be empty.

Suggestions?

[1ec5]

In this StackOverflow post, I outlined a simple way to insert layers between roads and labels/shields/icons, which is a pretty common use case. That SO post was written in Swift for the Mapbox iOS SDK, but the same approach should be feasible in GL JS as well: iterate over all the layers in the map style, inserting the route layer above (after) the first non-symbol layer you find – that is, below (before) the last symbol layer you find.

[mb12]

@1ec5 my understanding from documentation and trying this on GL JS is that the current behavior of addLayer(before) depends on map state. Specifically if no buckets have been created for the layer before, the layer is added at the end. This makes it really difficult to use addLayer (before) to reliably model scenarios like this without manually modifying style.json. AddLayer (before/after) works reliably for modeling only when working with Geojson sources.

Is GL native implementation of addLayer more reliable when working with vector sources!?

[1ec5]

As far as I know, in the native SDKs, inserting a layer before or after a vector layer should work reliably as long as you do so after the style finishes loading. (In fact, the SDKs actively prevent you from adding layers before that point.) However, the native SDKs’ events don’t always correspond to the events in GL JS, so I don’t know if there’s an additional requirement that buckets are created beforehand.

[mourner]

@mb12 if you experience inconsistencies in how layer order is handled, could you please set up a minimal JSFiddle test case that reproduces this? Sounds like a bug.

[1ec5]

With nested styles or “style components”, we’d be able to revisit mapbox/mapbox-gl-native#5665 so that only components affected by runtime styling would become static, while other components would continue to refresh periodically. It turns out that some developers have expected the style to continue to refresh despite making runtime styling changes.

/cc @eschow

[pablo-slingshot]

"Nested" styles helps solve the common preserve-my-custom-layers-while-switching-"basemaps" problem.

@lucaswoj Is there a workaround for this?

[lucaswoj]

@pablo-slingshot Not really. My best recommendation is create a createMapboxStyle function which internally fetches some basemap style and concatenates your custom layers. All style mutations should be expressed as arguments to createMapboxStyle (as opposed to using methods like setPaintProperty). This approach works nicely with the "smart" diffing algorithm in Mapbox GL -- even though you're passing a "new" style, it'll intelligently animate in the changes when possible.

[rogeraustin]

@lucaswoj I am considering something like your suggestion for merging my custom style (using my own layers and sources) with a standard basemap style. The potential issue I see is that I have a custom sprite and the style spec allows only one sprite. The idea of dynamically composing two sprites into one does not appeal (although I could potentially do it in the server code that generates my custom sprite on demand). Do you have any suggestions? Perhaps the sprite property of the style could accept an array of URLs and then either rely on the image names being unique across sprites, or add a layer property icon-sprite to disambiguate where needed (where icon-sprite specifies the last part of the URL path).

[lucaswoj]

@rogeraustin Merging sprites and glyphs is definitely a larger engineering problem but possible with the existing APIs (addImage, transformRequest)

[stevage]

@lucaswoj Are you able to elaborate on that? A recent question came up on StackOverflow on this topic: https://stackoverflow.com/questions/59738350/combine-more-than-one-style-in-a-map

Would you mind spelling out the specifics of combining two font glyphs or two icon sprites?

[leogermani]

I'm currently trying to combine styles in a map and facing some challenges.

I'm aware of the limitation that a style can have only one glyph set, but even with styles from the same account I'm having trouble adding some symbol layers.

My general approach is:

1. Add a style
2. Load the second style details with a request to https://api.mapbox.com/styles/v1/{id} (let's call this response styleDefinition)
3. Iterate over styleDefinition.sources and add them using addSource to the current map (prefixing the name to avoid conflict)
4. Iterate over styleDefinition.layers and add them using addLayer to the current map (prefixing the id to avoid conflict, and replacing the source to the modified source name in step 3. Note that all layer properties, such as layout, are just passed as they were received.

I'm having two major issues

1. Text layer (type symbol): They work fine as long as they are added with no layout.visibility information. If I try to add them hidden by default, I'm not able to make them visible again

2. Icon layer (type symbol): Icons never show up. One thing I notice, comparing when I add this same Style as the base style of the map, is that the layer.layout property looks misconfigured.

Layer when added as Style:

Layer when added via my approach to add additional Style.

Hope the issue is clear and would be really help if someone could point to a possible solution.

Thank you very much

[rogeraustin]

@leogermani - my advice would be:

1. Get the current style object from the map using Map.getStyle. This is a plain old data object.
2. Merge the source and style layers from the new style into this object.
3. Call Map.setStyle with this updated style.

setStyle is 'smart' and will do the minimal updates necessary to correctly update the map.

Note that both styles must use the same sprite. If not, icons from the second style will not display (unless they also happen to be in the first sprite). This may be what you are seeing.

If you have different sprites, then you should be able to add the images from the second sprite to the map using Map.addImage as @lucaswoj suggests above (I have not tried this). This would require you to load the sprite image, load and parse the corresponding sprite.json file to find all the individual images within it, carve these out of the sprite image using createImageBitmap, and add them using Map.addImage, renaming them as needed for uniqueness (and updating any style layers that use them to use the new names). You would also need to decide whether to use the normal or hi-res sprite.

As I say, I have not done this and it seems it would be pretty painful - hence my suggestion above that the style spec could be updated to allow the sprite property to take an array of sprite urls and allow an icon-sprite property to be set on a style layer to disambiguate image names (if needed). This would enable proper merging of styles at the 'plain old data' level.

[leogermani]

Hi @rogeraustin ,

Thank you for your reply.

I tried what you suggested and got exactly the same outcome. Same behavior, same object as my original screenshot.

Here is some code:

// styleDefinition is the response from the API
let currentStyle = map.getStyle();

Object.entries(styleDefinition.sources).forEach( ([source_key, source]) => {
    //map.addSource( uniquePrefix + '_' + source_key, source );
    currentStyle.sources[uniquePrefix + '_' + source_key] = source;
});

styleDefinition.layers.forEach(layer => {

    layer.id = uniquePrefix + '_' + layer.id;

    if ( layer.source ) {
        layer.source = uniquePrefix + '_' + layer.source;
        //map.addLayer( layer );
        currentStyle.layers.push(layer);
    }

});

map.setStyle( currentStyle );

How can I check and make sure "they are using the same sprites". Both styles have the same value in style.glyphs.

Thanks!

[rogeraustin]

style.glyphs is for the font stack. You need to compare the values in style.sprite