MVT (Mapbox Vector Tile ~ Protobuf vector tiles) support

[mapcentia]

GC2 does now support protobuf vector tiles. We need to do some tentative testing of protobuf tile support in Vidi. Ideal we should be able to choose between raster tiles, vector (current options) and vector tiles (new). There is a Leaflet plugin for vector tiles: https://github.com/Leaflet/Leaflet.VectorGrid

https://gc2.mapcentia.com is not upgraded to vector tiles, so you need to test with https://test.gc2.io

I've made a small demo with vector tiles from test.gc2.io using MapBox GL: https://github.com/mapcentia/mvt-demo

[sashuk]

@mapcentia Could you please specify if vector tiles should be a separate representation of data (that has same importance as tile and vector, separate controls, etc.) or it should be an option for vector layers (for example, toggle Display layer using vector tiles)?

[sashuk]

@sashuk Keep working in https://github.com/sashuk/vidi/tree/vector-tiles

[mapcentia]

@sashuk Importance of vector tiles? Yes, I would like it to be a thirds option of available Layer Types. But we also need some way of defining a style document. The use case for vector tiles is somewhat more similar to raster tiles than vector layers.

We also need to be able to use vector tiles for base layers. This may be the most typical use of vector tiles. Maybe a separate issue for vector tile base layers?

[sashuk]

@mapcentia Got it, thanks! The #252 was created.

[sashuk]

@mapcentia Speaking of vector tiles being the third option - am I correct that GC2 backend will have vector tiles as an option for layers as well? Or vector tiles layers should be automatically created for those layers that are plain vector ones?

[mapcentia]

@sashuk Yes, you're corrct. There is now a third option: Vector tile => mvt

[sashuk]

@mapcentia Got it, thanks!

[sashuk]

@mapcentia Seem like when choosing Both in GC2, the vidi_layer_type = tv in meta object (though it probably should be mvt + v + t = mvtvt)

[mapcentia]

@sashuk Yes, it should be All > mvtvt. Or maybe All > all. WebGL is also becoming an option.

[sashuk]

@mapcentia Got it, thanks! Should I modify the GC2 to have 4 layer types available? There probably should be a multi-choice checkbox, as it is hard to list all combinations using <select/>.

The integration of new layer types in https://github.com/sashuk/vidi/tree/vector-tiles is coming to an end, so it will be great to have the GC2 able to set different layer type combinations for testing purposes.

[mapcentia]

@sashuk Yes, a multi-choice checkbox is the way to go.

[sashuk]

@mapcentia Got it, thanks! I will get to GC2 modification right away then.

[sashuk]

@mapcentia Should the vector tiles layers support the GeoJSON data sources or only the mvt ones?

[mapcentia]

@sashuk I do not fully understand the question. MVT is a new API in GC2, and have nothing with GeoJSON?

[sashuk]

@mapcentia Vector tiles can be created from GeoJSON as well (http://leaflet.github.io/Leaflet.VectorGrid/vectorgrid-api-docs.html#vectorgrid-slicer), so no need in this functionality?

[mapcentia]

@sashuk no need for that. Every layer can be requested as either GeoJSON or MVT. So we will always prefer the MVT service.

[sashuk]

@mapcentia So, the data source for vector layers is GeoJSON (api/sql requests), the data source for vector tile layers is MVT API like in https://github.com/mapcentia/mvt-demo/blob/master/simple.json#L7. Got it, thanks!

[sashuk]

@mapcentia Am I correct that in order to vector tile layer to have latest data changes the cache should be reset through the GC2 backend, or there should be the mechanism of doing it on the frontend?

[mapcentia]

@sashuk The tile bust mechanism is on the back-end and working already for raster tiles. MapCache is used for both raster and vector tiles, so there is no difference between the two types other than the format.

As for raster, the vector tiles can also be requested as "Single tile" directly from the WMS back-end. So vector tiles should also be able to switch to Single Tile mode (if its doable in Vidi to display single vector tile).

Vector tiles from MapCache (gmap): https://test.gc2.io/mapcache/test/gmaps/public.skov.mvt/{z}/{x}/{y}.mvt

Vector tiles from MapCache (TMS): http://test.gc2.io/mapcache/test/tms/1.0.0/public.skov.mvt/9/268/355.mvt

Vector tile from WMS: http://test.gc2.io/wms/test/public?LAYERS=public.skov&TRANSPARENT=TRUE&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&STYLES=&**FORMAT=mvt**&SRS=EPSG%3A3857&BBOX=978393.96191406,7631472.9029297,1017529.7203906,7670608.6614062&WIDTH=256&HEIGHT=256

Its only the FORMAT=mvt, which differ from a raster WMS request

[sashuk]

@mapcentia Got it, thanks!

[sashuk]

Current state of things:

[sashuk]

@sashuk The tile bust mechanism is on the back-end and working already for raster tiles. MapCache is used for both raster and vector tiles, so there is no difference between the two types other than the format.

As for raster, the vector tiles can also be requested as "Single tile" directly from the WMS back-end. So vector tiles should also be able to switch to Single Tile mode (if its doable in Vidi to display single vector tile).

Vector tiles from MapCache (gmap): https://test.gc2.io/mapcache/test/gmaps/public.skov.mvt/{z}/{x}/{y}.mvt

Vector tiles from MapCache (TMS): http://test.gc2.io/mapcache/test/tms/1.0.0/public.skov.mvt/9/268/355.mvt

Vector tile from WMS: http://test.gc2.io/wms/test/public?LAYERS=public.skov&TRANSPARENT=TRUE&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&STYLES=&**FORMAT=mvt**&SRS=EPSG%3A3857&BBOX=978393.96191406,7631472.9029297,1017529.7203906,7670608.6614062&WIDTH=256&HEIGHT=256

Its only the FORMAT=mvt, which differ from a raster WMS request

Seems like in order to use the non-cached version of MVT layers the URL should look like this:

https://test.gc2.io/wms/aleksandrshumilov/public?mode=tile&tilemode=gmap&tile=4989+4252+13&layers=public.testingvectortiles_polygons&format=mvt&map.imagetype=mvt&

If only FORMAT field is changes, then there is still a problem of how to feed the WMS request URL to the LeafletVectorGrid (as it has to generate URL's by itself using the {x}/{y}/{z} notation). Request paramters are taken from https://mapserver.org/development/rfc/ms-rfc-119.html#sample-mapserver-requests

[sashuk]

Cache busting for VectorTile layers was implemented (for example, displaying VectorTile layer after its Vector representation was edited in editor):

[sashuk]

@mapcentia Seems like WMS filters are not working:

Request: https://test.gc2.io/wms/aleksandrshumilov/public?filters={%22public.testingvectortiles_polygons%22:[%22id%20%3E%201%22]}&service=WMS&request=GetMap&version=1.1.1&layers=public.testingvectortiles_polygons&styles=&format=image%2Fpng&transparent=true&tileSize=256&srs=EPSG%3A3857&width=1160&height=803&bbox=4363197.557831503,-768688.9749498754,4385364.296034204,-753344.2415216257

The Request's filters section: filters={%22public.testingvectortiles_polygons%22:[%22id%20%3E%201%22]} (decoded as filters={"public.testingvectortiles_polygons":["id > 1"]})

Expected result: only 1 polygon (table data below)

Actual result:

WMS filters format is used according to the https://github.com/sashuk/vidi/issues/224

Currently investigating this issue, as WMS filters are not working for VectorTiles as well

---

For reference: it is normal not to work for VectorTiles, as some changes are required in https://github.com/mapcentia/geocloud2/blob/master/app/controllers/Wms.php#L111, but still no reason why it is not working for regular WMS layers

[mapcentia]

@sashuk The WMS filtering is for now only implemented for layers setup with QGIS-server. Not with MapServer. I'm looking into how the latter also can be supported.

[sashuk]

@mapcentia What if use SLD filtering (https://mapserver.org/ogc/sld.html#specific-sld-elements-supported) and generating SLDs on-the-fly for MapServer requests? This way data can be filtered

[mapcentia]

@sashuk Esri describe the problem with using SLD for filtering here: http://enterprise.arcgis.com/en/server/latest/publish-services/linux/filtering-features-using-the-layerdefs-parameter-in-wms-requests.htm The problem is that you have to recreate the symbologies in the SLD and there will be some styles which can't be defined in SLD.

[sashuk]

@mapcentia Thanks for the information!

[sashuk]

@mapcentia The MVT layers are now supported, however, without filtering. Should the filtering be implemented later in the separate issue, so this one could be closed?

[mapcentia]

@sashuk closing. I'll open a new one about filtering