Add support for PMTiles

[giswqs]

PMTiles is a single-file archive format for tiled data. A PMTiles archive can be hosted on a commodity storage platform such as S3, and enables low-cost, zero-maintenance map applications that are "serverless" - free of a custom tile backend or third party provider.

Currently, it is challenging to render large vector datasets with ipyleaflet. PMTiles can be a great option for rendering large vector datasets with ipyleaflet. The folium-pmtiles package supports rendering PMTiles with folium. See below for an example.

This PR tries to add ipyleaflet support for PMTiles. However, I have very limited JavaScript knowledge. I need your help. Thanks.

@martinRenou @davidbrochart @jtmiclat @bdon

https://github.com/protomaps/PMTiles/discussions/209 https://github.com/jupyter-widgets/ipyleaflet/issues/1134

import folium
from folium.elements import JSCSSMixin
from folium.map import Layer
from jinja2 import Template

class PMTilesMapLibreLayer(JSCSSMixin, Layer):
    """Based of
    https://github.com/python-visualization/folium/blob/56d3665fdc9e7280eae1df1262450e53ec4f5a60/folium/plugins/vectorgrid_protobuf.py
    """

    _template = Template(
        """
            {% macro script(this, kwargs) -%}
            let protocol = new pmtiles.Protocol();
            maplibregl.addProtocol("pmtiles", protocol.tile);

           {{ this._parent.get_name() }}.createPane('overlay');
           {{ this._parent.get_name() }}.getPane('overlay').style.zIndex = 650;
           {{ this._parent.get_name() }}.getPane('overlay').style.pointerEvents = 'none';

            var {{ this.get_name() }} = L.maplibreGL({
            pane: 'overlay',
            style: {{ this.style|tojson}}
            }).addTo({{ this._parent.get_name() }});

            {%- endmacro %}
            """
    )
    default_css = [
        ("maplibre_css", "https://unpkg.com/maplibre-gl@2.2.1/dist/maplibre-gl.css")
    ]

    default_js = [
        ("pmtiles", "https://unpkg.com/pmtiles@2.5.0/dist/index.js"),
        ("maplibre-lib", "https://unpkg.com/maplibre-gl@2.2.1/dist/maplibre-gl.js"),
        (
            "maplibre-leaflet",
            "https://unpkg.com/@maplibre/maplibre-gl-leaflet@0.0.17/leaflet-maplibre-gl.js",
        ),
    ]

    def __init__(self, url, layer_name=None, style=None, **kwargs):
        self.layer_name = layer_name if layer_name else "PMTilesVector"

        super().__init__(name=self.layer_name, **kwargs)

        self.url = url
        self._name = "PMTilesVector"

        if style is not None:
            self.style = style
        else:
            self.style = {}

m = folium.Map(location=[43.7798, 11.24148], zoom_start=13)
pmtiles_url = "https://open.gishub.org/data/pmtiles/protomaps_firenze.pmtiles"
pmtiles_layer = PMTilesMapLibreLayer(
    "folium_layer_name",
    overlay=True,
    style={
        "version": 8,
        "sources": {
            "example_source": {
                "type": "vector",
                "url": "pmtiles://" + pmtiles_url,
                "attribution": '<a href="https://protomaps.com">Protomaps</a> © <a href="https://openstreetmap.org/copyright">OpenStreetMap</a>',
            }
        },
        "layers": [
            {
                "id": "buildings",
                "source": "example_source",
                "source-layer": "landuse",
                "type": "fill",
                "paint": {"fill-color": "steelblue"},
            },
            {
                "id": "roads",
                "source": "example_source",
                "source-layer": "roads",
                "type": "line",
                "paint": {"line-color": "black"},
            },
        ],
    },
)
m.add_child(pmtiles_layer)
folium.LayerControl().add_to(m)
m

[giswqs]

Another JS library that can be used for this: https://github.com/protomaps/protomaps-leaflet

[jtmiclat]

Got a working protomaps-leaflet in the following commit: https://github.com/jupyter-widgets/ipyleaflet/commit/cafc323ab3cdceae4a47e883859f51f4c46cfd82

But facing the same issue that there isn't a simple way styling it

[giswqs]

Amazing work! That's one big step forward. I am excited

[jtmiclat]

@giswqs Pushed a working version with styling to https://github.com/jtmiclat/ipyleaflet/tree/pmtiles. Feel free to merge that branch here. I learned that protomaps-leaflet had a function to convert simple mapbox styles to protomap styles. Might propagate that change to folium-pmtiles

Example usage:

from ipyleaflet import Map, PMTilesLayer

m = Map(center=[43.7798, 11.24148], zoom=13)
vl = PMTilesLayer(url="https://pmtiles.jtmiclat.me/protomaps(vector)ODbL_firenze.pmtiles", 
    style={
        "layers": [
            {
                "id": "landuse",
                "source": "example_source",
                "source-layer": "landuse",
                "type": "fill",
                "paint": {"fill-color": "black"},
            },
            {
                "id": "roads",
                "source": "example_source",
                "source-layer": "roads",
                "type": "line",
                "paint": {"line-color": "steelblue"},
            },
        ],
    })
m.add_layer(vl)
m

Based on https://github.com/protomaps/protomaps-leaflet/issues/112 this function will be removed in the future though!

[giswqs]

@jtmiclat Sorry for the delay! I have incorporated your code into this PR. Thank you very much for your help with this.

@martinRenou The new feature allows ipyleaflet to visualize large vector datasets. It will greatly benefit the geospatial community. The unit tests have all passed. Please review it when you have time.

[giswqs]

Just added a notebook example for visualizing a 1.1 GB PMTiles.

from ipyleaflet import Map, basemaps, PMTilesLayer

m = Map(center=[52.963529, 4.776306], zoom=7, basemap=basemaps.CartoDB.DarkMatter, scroll_wheel_zoom=True)
m.layout.height = '600px'

vl = PMTilesLayer(url="https://storage.googleapis.com/ahp-research/overture/pmtiles/overture.pmtiles", 
    style = {
        "layers": [
            {
                "id": "admins",
                "source": "example_source",
                "source-layer": "admins",
                "type": "fill",
                "paint": {"fill-color": "#BDD3C7", "fill-opacity": 0.1},
            },
            {
                "id": "buildings",
                "source": "example_source",
                "source-layer": "buildings",
                "type": "fill",
                "paint": {"fill-color": "#FFFFB3", "fill-opacity": 0.5},
            },
            {
                "id": "places",
                "source": "example_source",
                "source-layer": "places",
                "type": "fill",
                "paint": {"fill-color": "#BEBADA", "fill-opacity": 0.5},
            },
            {
                "id": "roads",
                "source": "example_source",
                "source-layer": "roads",
                "type": "line",
                "paint": {"line-color": "#FB8072"},
            },
        ],
    })
m.add(vl)
m

https://github.com/jupyter-widgets/ipyleaflet/assets/5016453/45985dc8-6aa3-4159-925d-00afb2929a58

[giswqs]

Can one of the maintainers review and merge this PR?

[giswqs]

It would be great if this PR can be included in the next release