Vector Tiles on osm.org

[bhousel]

From @bhousel on May 21, 2018 19:49

I've been reviewing the Top 10 Tasks list, and noticed that several of the issues reference vector tiles as a possible way forward.

• Localized Map Rendering - https://github.com/osmlab/osm-planning/issues/9
• Clickable POIs - https://github.com/osmlab/osm-planning/issues/16

This is also something that pops up for discussion sometimes here in this repo (#1672 #1623 #1316)

Let's start a discussion about what it would look like to add a vector tile option to openstreetmap.org! Even if it's just a trial or staged rollout available only in some areas, I think it would be great to explore vector tiles as a path forward on some of these issues.

I think we all agree that if something is on the Top 10 Tasks list, it's important to enough to people that it's worth doing. It's time! ✨

Copied from original issue: openstreetmap/openstreetmap-website#1875

[systemed]

@systemed Is there something happening with it?

Yes. Life gets in the way so less quickly than I'd have hoped! But hope to have a v0.000001omega for people to start hacking on in the next few weeks.

[mboeringa]

@mboeringa : I have to disagree with your proposal. Besides usability issues, this kind of usage pattern will simply create way too much load on today's Overpass API instances, and requests will be very quickly discarded with HTTP 429 Too many queries error messages due to rate limiting.

@mmd-osm Thanks for your insights, however, could you elaborate a bit more on why it is so problematic?

Mark already limited viewing of the POI by forcing you to zoom-in on the map before data is being displayed. His tool apparently isn't blocked because of over-usage right now. In addition, there are other tools doing about the same thing, which are neither blocked, like OpenLevelUp, which also limits you to viewing data only on high zoom. I realize you are right that putting such tools on the main website, will generate a lot more exposure and usage, but such tools are already there.

Even the main website currently has an option to request data from what I guess is also the API: the "Map data" checkbox in the map layers menu. And of course editors like iD need to request data for each pan and zoom.

I am just trying to get a feeling of why this would be so problematic in the context of all other things already going on, and base tools of OSM already plugging in on the API.

Anyway, of course it is good to wait for upcoming optimizations before considering integration of any such tools if this is a real concern.

However, as to vector tiles serving this same clickable POI function: I do think vector tiles will only have limited value for this in the context of OSM's vast data model.

Just imagine you are a health care professional working for HOT and just having entered a vast amount of detailed health care keys for the health facilities in your district. You now want to view the metadata / expose it by clicking on the POI on the map based on vector tiles. Most likely: bad luck! Unfortunately, since the vector tiles are pre-cooked and highly optimized in terms of data included, they almost certainly won't contain any of the 50+ keys you may have entered, since these keys have no relevance for the tile provider and the limited set of keys needed for the actual styling. It will require creating an entirely dedicated custom vector tile set to expose the data...

Requesting data in the way OpenPoiMap or the "Map data" option on OSMs main website do, doesn't suffer from this limitation, you will always have access to all keys and relevant data.

[mmd-osm]

Even the main website currently has an option to request data from what I guess is also the API: the "Map data" checkbox in the map layers menu. And of course editors like iD need to request data for each pan and zoom.

This is all served either by the Rails port or cgimap, it has nothing to do with Overpass API.

His tool apparently isn't blocked because of over-usage right now.

The main issue is that once you activate something like this on the main page, we're not talking about 10 users, but thousands of users, and they want to see those details even after panning the map a few times without getting "Too many requests" errors, which is the case with OpenPoiMap. By the way, once you enable a few layers you will experience this issue very quickly- the code isn't optimized to merge several layers into as few requests as possible, hence already creates way too many queries the way it is implemented today.

[mboeringa]

@mmd-osm , thanks for your additional insights and comments, this explains the problem you pointed out well.

[kocio-pl]

So it means that we could probably have clickable POIs without vector tiles, if somebody would design a proper user interface using underlying technology we have today. Nice to know, yet some coder would be needed to do also that.

Having vector tiles would be nice anyway - they match the "unique" soft requirement. It would enable hacking the style, because there's a data file - it's hard to maintain, but easy to play with.

[mmd-osm]

@kocio-pl : I don't know how you came to this conclusion. In case this is somehow backed by the openpoimap story mentioned by @mboeringa, be assured that this is quite some time away for the backend side to get performance improvements in place (yes, Roland is working on GDPR, and there's no capacity for anything else). The "using underlying technology we have today" point is pretty much moot at the moment.

[kocio-pl]

I have not said anything about performance (I don't know how much people use data layer today and how much power would be needed for dynamic POIs), I just noticed that we don't need special vector stack for showing dynamic POIs. Do you think this is unsuitable tool for this task?

@systemed That would be great! :smile:

[mmd-osm]

Do you think this is unsuitable tool for this task?

In the short term it's definitively not feasible via Overpass API, assuming dynamic POIs are enabled by default for every osm.org visitor. This may change some time in 2019, or maybe even later, depending on when performance improvements get integrated. Still it's completely unclear what hardware requirements will look like then.

NB: As mentioned earlier, the data layer is served via an API 0.6 /map call (not Overpass API), and returns just about all data in a bounding box (not just POIs). I also doubt if this is the right approach for dynamic POIs.

[ImreSamu]

it's too bad that @skmoore's talk didn't get recorded

now the video is online:

youtube link: "Breaking Down Silos: OSM and Esri side by side - Steve Moore, Esri"
Published on 25 Oct 2018 https://www.youtube.com/watch?v=rTuFf7Evy2s

and inside - the program:

• https://2018.stateofthemap.us/program/breaking-down-silos-osm-and-esri-side-by-side.html

[kocio-pl]

Thanks, it was good to watch it. I have a question for Esri - do you plan to add choosing languages on your vector map style?

[jgravois]

do you plan to add choosing languages on your vector map style?

its definitely something we'd like to do, but its not on the immediate roadmap and R&D would be required to determine exactly how we could make it work.

[pnorman]

I've been thinking about this, and I don't think there's any operations work to be done right now. If someone has a new vector tile layer they want on osm.org, be it server- or client-side rendered, they could request it and it would be evaluated against the guideline for new tile layers.

If someone has an open-source style they want the OSMF to host, they could bring it to the OWG.

Until either of these happen, I just don't see any OWG or sysadmin work.

[bhousel]

Ok @pnorman - If there is nothing actionable on this ticket, we can close it. I don't have the energy to push for this right now.

[pnorman]

I don't have the energy to push for this right now.

I think all the ops want vector tile, it's just there's nothing ops-related right now I can see to do.

[mnalis]

• related: https://github.com/openstreetmap/operations/issues/565

[mnalis]

• ref: https://github.com/openstreetmap/openstreetmap-website/pull/4042