Support client-side rendered featured layers

[pnorman]

The OWG has had a couple of submissions for featured layers that involve client-side rendered featured layers.

We should figure out how to support them. The requirements I see are

• other layers work when a user's setup doesn't support a particular client-side layer (e.g. if webgl is disabled)

• we aren't locked into only supporting one client-side rendering engine

• large libraries aren't loaded until a user needs it

• client-side layer support in the layer switcher bar

• support for fractional zooms on client-side layers

The client-side rendered layers we've had interest from are using Mapbox GL, but like I mentioned, we need to be able to support multiple client-side rendering engines in the future

Two methods mentioned are

• use mapbox-gl-leaflet, or
• shift to Mapbox GL JS.

I don't know enough to know if there are other options. Shifting to Mapbox GL JS seems like more work, and would likely make it more difficult to use other client-side rendering engines, so it's probably not the way to go.

[ra0e]

From my understanding there are no other options. I'd suggest to goo Mapbox GL JS too, since it is faster and future proof. However it is much more work to migrate to vector tile based rendering.

[pnorman]

The work to migrate a style to a vector tile based rendering doesn't matter here, this issue is about supporting client-side rendering on the website. Migrating OpenStreetMap Carto or changing what the OSMF hosts with tile.openstreetmap.org isn't a website issue.

[systemed]

Mapbox GL Native (i.e. in-app library for Android/iOS) has just gone closed-source. Before considering Mapbox GL JS for osm.org, we would presumably need a commitment that this is going to remain open source. /cc @mikelmaron

[mmd-osm]

OL can do vector tiles these days. IDK if they would meet all requirements for client side rendering: https://openlayers.org/en/latest/examples/mapbox-vector-tiles.html

[pnorman]

Mapbox GL Native (i.e. in-app library for Android/iOS) has just gone closed-source. Before considering Mapbox GL JS for osm.org, we would presumably need a commitment that this is going to remain open source. /cc @mikelmaron

Mapbox GL JS has gone un-open source. This excludes them from consideration here. Options which consume Mapbox GL JSON styles are still an option, as are other rendering engines.

[tomhughes]

https://github.com/mapbox/mapbox-gl-js/issues/10162 is the reference for the mapbox gl reverse ferret.

[jvolker]

How about https://github.com/maplibre/maplibre-gl-js?

MapLibre GL is a community led fork derived from mapbox-gl-js prior to their switch to a non-OSS license.

[grischard]

@pnorman could this issue be re-titled "Migrate from Leaflet to maplibre-gl-js"?

[pnorman]

@pnorman could this issue be re-titled "Migrate from Leaflet to maplibre-gl-js"?

There's an implementation question there that the maintainers would need to weigh in on.

One implementation option is to completely replace Leaflet with Maplibre GL JS. This would

• presumably speed up the data layer
• minimize total data transferred if someone uses both a raster layer and a vector layer
• minimize code complexity
• require WebGL for most of the site to work
• lock us in to only client-side rendering that Maplibre GL JS supports

The other option is to use Leaflet for raster layers, and only use Maplibre GL JS for vector layers. This would

• require data layer implementations for Leaflet and Maplibre GL JS
• minimize time to map load for raster layers because Leaflet is smaller
• keep the site working for users without WebGL
• keep the option open for different client-side rendering

My preference is for the first option, mainly for code complexity reasons. The latter would require our leaflet integration, a maplibre integration, and logic to switch between them. But making this decision isn't my call, it's a maintainer decision.

[tomhughes]

Well I think we need to know the impact of requiring WebGL but my initial reaction would be that requiring it is unlikely to be possible.

I think all my machines have it enabled now but it's not that long ago that I had it disabled on some browsers.

[pnorman]

Can we quantify what portion of traffic doesn't support WebGL with piwik?

[tomhughes]

I don't believe there are are any builtin reports for that.

We could add something to the site that flagged an indicator to piwik based on whether the browser looked like it was capable?

[pnorman]

I haven't used piwik/Matomo and I don't have access, so I'm not sure what's possible. caniuse reports good webgl coverage, but as you pointed out, for various reasons, webgl might be disabled on a browser that otherwise supports it.

[tomhughes]

I know Firefox at least blacklists various features depending on the exact OS/graphics card/driver in use so it's not as simple as whether the browser in theory supports it - whether any of the effects the level of WebGL needed here I don't know.

[pnorman]

Looks like https://github.com/sgiehl/piwik-plugin-DeviceFeatureWebGL will track webgl support which will get us the data needed to make an informed decision.

[tuukka]

Looks like https://github.com/sgiehl/piwik-plugin-DeviceFeatureWebGL will track webgl support which will get us the data needed to make an informed decision.

This is the library that MapLibre uses to check whether the browser supports what is needed, including specific WebGL and JavaScript features: https://github.com/mapbox/mapbox-gl-supported

My preference is for the first option, mainly for code complexity reasons. The latter would require our leaflet integration, a maplibre integration, and logic to switch between them. But making this decision isn't my call, it's a maintainer decision.

Could a simple integration using maplibre-gl-leaflet work as an upgrade path to consider? Like this: Load the Leaflet map as usual by default. Only if/when the user selects a client-side rendered layer (a vector background layer or features), load maplibre-gl-leaflet and MapLibre code dynamically and add it as a layer to the Leaflet map. The downside is a little less performance caused by how maplibre-gl-leaflet works. As an upside, this shouldn't require big changes to the existing code or much new code. https://github.com/maplibre/maplibre-gl-leaflet

[tomhughes]

https://github.com/mapbox/mapbox-gl-supported/blob/gh-pages/index.js#L162 seems like a huge red flag to me - if Brave won't work that is going to be an issue.

[pnorman]

According to https://github.com/mapbox/mapbox-gl-js/issues/10518#issuecomment-808934647 the issue is only in Brave's strict mode, which disables extensive parts of WebGL, and is known to break websites. I just installed brave, and it does support Maplibre GL JS by default, and https://mapbox.github.io/mapbox-gl-supported/ reports that it is supported.

I turned Brave's block fingerprinting to "Strict, may break sites" and it does exactly what it says - breaks sites :)

[HolgerJeromin]

Can we quantify what portion of traffic doesn't support WebGL with piwik?

Was added in https://github.com/openstreetmap/chef/commit/bf96ed36324669a5c261a34fd7903663ff11c2aa

[pnorman]

Based on this morning's stats (n=70k), 15% of visitors do not have WebGL support. I would say that we need to maintain current support for clients without WebGL. I'm also wondering how the difference in experiences will be if 15% of the users see a different set of layers.

[tuukka]

Do we have a list of the layers that this issue concerns? I'd like to understand better which ones are not feasible without WebGL (apart from vector background layers).

[tomhughes]

Well it doesn't concern any layers right now, but the long term goal is to move the default layer to a vector layer which, if we use client side rendering, would require webgl support on the client. Paul is saying that as things stand we probably need to maintain a non-vector alternative at least on the client side but that could be achieved with server side rendering of vector tiles.

The immediate proposal on this ticket is about adding support for vector layers along side our current layers and specifically to do that bu switching the toolkit we use to display maps to a GL based one.

I'm not familiar enough with maplibre-gl-js to know it is supports bitmap layers itself and whether if it does it requires WeBGL to display them?

One caveat I'll add is that Paul's data was based on only a few hours because the webgl data collection was broken until late last night but it's consistent with what I saw earlier in the week for a full day.

[lseelenbinder]

Hey all. (I'm on the board for MapLibre, so I'm happy to answer specific questions related to that.)

I'm not familiar enough with maplibre-gl-js to know it is supports bitmap layers itself and whether if it does it requires WeBGL to display them?

It does support bitmap layers, but as far as I understand it requires WebGL (the entire render pipeline is WebGL, and bitmap layers are uploaded to the GPU for rendering along with vector layers).

[tuukka]

Using maplibre-gl-leaflet in the way I described above still sounds to me the way to go here in the short term. Some map styles would be available only as raster tiles (if they haven't been ported to MapLibre styles), some would use MapLibre styles and be available as client-rendered vector tiles for those with WebGL opt-in and as server-rendered vector tiles for the rest using Tileserver GL: http://tileserver.org/

Then in the longer term when WebGL can be made a requirement, Leaflet could be dropped and MapLibre used directly without maplibre-gl-leaflet. After that, the infrastructure for server-rendered tiles could be potentially discontinued.

[lseelenbinder]

I concur with @tuukka. I think that's probably the most compatible option available.

[pnorman]

Do we have a list of the layers that this issue concerns? I'd like to understand better which ones are not feasible without WebGL (apart from vector background layers).

None. All of the featured layers are raster layers, and supported by Leaflet. This issue is about potential future featured layers that are client-side rendered. This issue is also not about replacing the standard layer, which is out of scope for the website repo.

---

The current number is 11% (n=207k), so there's a time-dependent component to lack of WebGL support. I considered if it might be influenced by users who are only hitting the oauth login flow or copyright page, which have no maps, but the numbers for the front page are higher with 15% of users not support WebGL (n=52k). The numbers are likely an undercount, as setups like Brave in strict mode which don't support WebGL are also unlikely to load piwik.

[pnorman]

I've got more detailed numbers of WebGL 1.0 presence of viewers of the site.

On August 5th, the matomo plugins were fixed. After that, it took up to 7 days for all users to be guaranteed to have the latest javascript in their cache, meaning numbers from the 13th and onwards have all users who loaded the matomo javascript with WebGL tests. The numbers are also stable, having shown behavior before then of growing usage as caches expired. This was the cause of some earlier numbers, although the general trend of which users had more support was accurate.

Over the 13th to 23rd, the average WebGL support for the front page was 98%, and the average support for users who are editors was 92%. To try to break down where the users are coming from without WebGL, I looked at some segments. Chrome users are 93%, Firefox users are 94%, and Windows users are 94%. For the purposes of Matomo's identification, Chrome users are almost entirely Windows, Mac, and Linux users. Android users normally are classified as Chrome Mobile.

I speculate that the percentage of WebGL users is lower among desktop users than mobile users, as that seems to be the only variable that would correlate with those breakdowns. I'm adding segments to test this, but it doesn't seem critical for this issue.

There is a weak day-to-day variation in the numbers, with support being lowest on Friday at 91% among editors, but I would want more data before coming to any strong conclusions there. Similar correlations with date are found among all segments.

All of these numbers are among users who load piwik/matomo. Certain setups (e.g. Brave browser in strict mode) will load neither.

---

The rest of this is more opinion-based. At 98% of users, I think it's okay to have a WebGL layer as a default layer, but with editors at 90%, I think we need to maintain support for non-WebGL users, likely with maplibre-gl-leaflet. The priority of the site is to support people editing, and desktop users are more likely to be editors.

I am skeptical that the numbers will change significantly in the future, but we can see in a few years if they have changed.

I believe these are the first published numbers on real-world WebGL support, rather than theoretical support by browsers, ignoring drivers and other issues. The latter are not reliable. Interestingly, browser support numbers suggest higher desktop support (99.52%) than mobile (97.79%) support, but we observe the reverse.

The numbers suggest that for someone developing a browser-based OpenStreetMap editor targeting mainly desktop users, excluding users without WebGL excludes more users than something like excluding Safari users.

[systemed]

^ cc @tyrasd for SotM discussion

[planemad]

One limitation with Maplibre/Mapbox GL is the lack of support for proper rendering of Indic scripts which require complex text shaping. This has been a long standing paint point in the OSM India community https://github.com/osm-in/openstreetmap.in/issues/14#issuecomment-649213986

[tomhughes]

Well unless there is some alternative that does support text shaping that's not really very useful information, other than to make us aware of the limitation. It's not like we're going to go and write a new toolchain with text shaping support ourselves.