Migration to vector tiles

[kocio-pl]

Follow up to https://github.com/gravitystorm/openstreetmap-carto/issues/1975#issuecomment-384858383

As discussed privately among team members, it's good to migrate osm-carto to be rendered with vector tiles. To allow as smooth and painless migration path as possible, it should be ready for server side raster tiles rendering. It should be non-disruptive change for clients while allowing to start developing client side rendering and testing vector tiles in general.

This is not a fresh and untested approach, @gravitystorm has been using it for his own styles for a few years already - you can watch his presentation about it from SotM EU 2014:

https://www.youtube.com/watch?v=NoLJHgqDCzc

There's also a vector fork of osm-carto by @rory (also his presentation from SotM 2016):

https://github.com/geofabrik/openstreetmap-carto-vector-tiles https://2016.stateofthemap.org/2016/converting-a-regular-carto-project-to-vector-tiles-osmcarto-case-study/

However this fork is quite old and happened before our v4.x database migration. It's not clear now how useful it could be, other than being proof of concept.

This ticket is about coordination of the effort, more specific tickets might be needed for subtasks and technical background (like choosing the proper software stack) is yet to be discussed and tested.

[Bibi56]

@kocio-pl as said in PostGIS 3.1.0alpha2.

[ZeLonewolf]

A migration to vector tiles is a complete do-over of the technology stack, even if all the cartographic decisions remain the same. IMHO that's too big of a change to be executed within this repository (not to mention the name "openstreetmap-carto" no longer makes sense as a repo name -- it's really "openstreetmap-vector" or something along those lines). A vector version of this style should really start from a new repo and start with some fundamental decisions about what technology stack to build and then iteratively build up the features until they match.

I note the following points which may be of interest to folks looking for an update on vector tile work, and not otherwise mentioned on this thread:

• @pnorman built a partial implementation of the osm-carto style on a vector stack at https://github.com/pnorman/openstreetmap-cartographic (public demo) which could serve as a starting point.
• The OWG under openstreetmap/operations#565 is seeking to stand up a community vector tile server that would support "community style development" which may align with a vector version of this style.

[jeisenbe]

that's too big of a change to be executed within this repository

That is a good point. Should we close this issue, or is there anything still to be done in this style which would be useful for a theoretical vector tile alternative?

[feludwig]

Hello, I have been playing around with @pnorman's osm-carto style vector reimplementation, and the whole rendering stack (I recently wrote https://github.com/feludwig/pgsql-omt-schema, and a little demo https://feludwig.github.io/pgsql-omt-schema) And I was thinking, could the xml stylesheet not be transpiled to sql+json for client-side vectortiles ? Assuming client-side rendering (eg with maplibre gl js) of mvt/pbf vector tiles, two configuration files would be needed :

• json stylesheet client-side
• sql functions server-side (PostGIS has a ST_AsMVT(), and pg_tileserv just serves any sql function that accepts z,x,y coordinates as a vector tiles set)

The sql functions would be easy as a prototype: just take the table from mapnik xml.

• One difficulty is geometry simplification (happens automatically with ST_AsMVTGeom(), but maybe sprinkle some ST_SimplifyPreserveTopology if the mvt tiles are too size-heavy)

The bigger challenge is to transpile all the xml s to mapbox-json style layers (there would be a lot of them) and invert the <Min/MaxScaleDenominator> s to "expressions", eg I mean :

"icon-color":["step",["zoom"],"blue",15,"red",17,"green"]

.

• Also needed is a "diff merge" algorithm for "uniting" rules over the same filter, but different fill-colors and any other differing properties I am playing around with a prototype, but woud there be some gotchas in the mapnik xml specification you can think of ?

I looked for the "migration to vector tiles" discussion, is this the correct issue to post into ?

[imagico]

I looked for the "migration to vector tiles" discussion, is this the correct issue to post into ?

This issue was created to discuss ideas and potential concepts to transfer/re-implement this CartoCSS map style for a different tool-chain as an initiative from within the OSM-Carto developer community. This discussion never went beyond the brainstorming stage, largely because none of the currently available tool-chains for client side rendering is really compatible with this project (both in terms of the cartography and in terms of the openly cooperative development model).

Several independent projects have meanwhile been started to create styles with client side rendering that are designed for some (usually superficial) similarity with OSM-Carto. The one that got closest so far is actually the earliest (the ESRI demo that was mentioned in the discussion above).

And I was thinking, could the xml stylesheet not be transpiled to sql+json for client-side vectortiles ?

I am not familiar with projects in that direction. I don't think this would be practically useful for a map style like OSM-Carto. Keep in mind this is nothing like the conversion of code between two low level turing-complete languages. This is about converting map design between two frameworks that are based on completely different overall paradigms.

And this is definitively the wrong forum for asking this kind of question.