Support pydeck-carto

[MarcSkovMadsen]

With Panel 1.4.0 the Deck.gl pane should support the Carto layout. Thus there would be hope that the pydeck-carto package would be supported.

It seems not. I end up experiencing

bokeh.core.serialization.SerializationError: can't serialize <class 'pydeck.types.function.Function'>

Reproducible example

• Create an account at https://carto.com/
• In the "Developer Section" create a "Built Application". You can use https://example.domain.com as "App url". Under advanced settings you should set "Application Type" to "Machine to Machine".
• Create a file carto_credentials.json

  {
  "api_base_url": "https://gcp-europe-west1.api.carto.com",
  "client_id": "INSERT HERE",
  "client_secret": "INSERT HERE"
  }

• Create a file app.py

import pydeck as pdk
import pydeck_carto as pdkc
from carto_auth import CartoAuth

# Authentication with CARTO
print("register")
carto_auth = CartoAuth.from_m2m("carto_credentials.json", use_cache=False)
print("done")

# Register CartoLayer in pydeck
pdkc.register_carto_layer()

# Render CartoLayer in pydeck
layer = pdk.Layer(
    "CartoLayer",
    data="SELECT geom, name FROM carto-demo-data.demo_tables.airports",
    type_=pdkc.MapType.QUERY,
    connection=pdkc.CartoConnection.CARTO_DW,
    credentials=pdkc.get_layer_credentials(carto_auth),
    get_fill_color=[238, 77, 90],
    point_radius_min_pixels=2.5,
    pickable=True,
)
view_state = pdk.ViewState(latitude=0, longitude=0, zoom=1)
deck=pdk.Deck(layer, map_style=pdk.map_styles.ROAD, initial_view_state=view_state)

import panel as pn
pn.extension("deckgl")
pn.pane.DeckGL(deck).servable()

• run

panel serve app.py --autoreload

• open http://localhost:5006/app
• Check the terminal

2024-04-04 18:40:12,469 error handling message
 message: Message 'PULL-DOC-REQ' content: {}
 error: SerializationError("can't serialize <class 'pydeck.types.function.Function'>")
Traceback (most recent call last):
  File "C:\repos\private\panel-140-social\.venv\Lib\site-packages\bokeh\server\protocol_handler.py", line 97, in handle
    work = await handler(message, connection)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\repos\private\panel-140-social\.venv\Lib\site-packages\bokeh\server\session.py", line 94, in _needs_document_lock_wrapper
    result = func(self, *args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\repos\private\panel-140-social\.venv\Lib\site-packages\bokeh\server\session.py", line 257, in _handle_pull
    return connection.protocol.create('PULL-DOC-REPLY', message.header['msgid'], self.document)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\repos\private\panel-140-social\.venv\Lib\site-packages\bokeh\protocol\__init__.py", line 131, in create
    return self._messages[msgtype].create(*args, **kwargs)  # type: ignore [attr-defined]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\repos\private\panel-140-social\.venv\Lib\site-packages\bokeh\protocol\messages\pull_doc_reply.py", line 82, in create
    content = PullDoc(doc=document.to_json())
                          ^^^^^^^^^^^^^^^^^^
  File "C:\repos\private\panel-140-social\.venv\Lib\site-packages\bokeh\document\document.py", line 752, in to_json
    roots = serializer.encode(self._roots)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\repos\private\panel-140-social\.venv\Lib\site-packages\bokeh\core\serialization.py", line 253, in encode
    return self._encode(obj)
           ^^^^^^^^^^^^^^^^^
  File "C:\repos\private\panel-140-social\.venv\Lib\site-packages\bokeh\core\serialization.py", line 278, in _encode
    return self._encode_list(obj)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "C:\repos\private\panel-140-social\.venv\Lib\site-packages\bokeh\core\serialization.py", line 326, in _encode_list
    return [self.encode(item) for item in obj]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\repos\private\panel-140-social\.venv\Lib\site-packages\bokeh\core\serialization.py", line 326, in <listcomp>
    return [self.encode(item) for item in obj]
            ^^^^^^^^^^^^^^^^^
  File "C:\repos\private\panel-140-social\.venv\Lib\site-packages\bokeh\core\serialization.py", line 253, in encode
    return self._encode(obj)
           ^^^^^^^^^^^^^^^^^
  File "C:\repos\private\panel-140-social\.venv\Lib\site-packages\bokeh\core\serialization.py", line 262, in _encode
    return obj.to_serializable(self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\repos\private\panel-140-social\.venv\Lib\site-packages\bokeh\model\model.py", line 534, in to_serializable
    super_rep = super().to_serializable(serializer)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\repos\private\panel-140-social\.venv\Lib\site-packages\bokeh\core\has_props.py", line 417, in to_serializable
    attributes = {key: serializer.encode(val) for key, val in properties.items()}
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\repos\private\panel-140-social\.venv\Lib\site-packages\bokeh\core\has_props.py", line 417, in <dictcomp>
    attributes = {key: serializer.encode(val) for key, val in properties.items()}
                       ^^^^^^^^^^^^^^^^^^^^^^
  File "C:\repos\private\panel-140-social\.venv\Lib\site-packages\bokeh\core\serialization.py", line 253, in encode
    return self._encode(obj)
           ^^^^^^^^^^^^^^^^^
  File "C:\repos\private\panel-140-social\.venv\Lib\site-packages\bokeh\core\serialization.py", line 278, in _encode
    return self._encode_list(obj)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "C:\repos\private\panel-140-social\.venv\Lib\site-packages\bokeh\core\serialization.py", line 326, in _encode_list
    return [self.encode(item) for item in obj]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\repos\private\panel-140-social\.venv\Lib\site-packages\bokeh\core\serialization.py", line 326, in <listcomp>
    return [self.encode(item) for item in obj]
            ^^^^^^^^^^^^^^^^^
  File "C:\repos\private\panel-140-social\.venv\Lib\site-packages\bokeh\core\serialization.py", line 253, in encode
    return self._encode(obj)
           ^^^^^^^^^^^^^^^^^
  File "C:\repos\private\panel-140-social\.venv\Lib\site-packages\bokeh\core\serialization.py", line 282, in _encode
    return self._encode_dict(obj)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "C:\repos\private\panel-140-social\.venv\Lib\site-packages\bokeh\core\serialization.py", line 343, in _encode_dict
    entries=[(self.encode(key), self.encode(val)) for key, val in obj.items()],
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\repos\private\panel-140-social\.venv\Lib\site-packages\bokeh\core\serialization.py", line 343, in <listcomp>
    entries=[(self.encode(key), self.encode(val)) for key, val in obj.items()],
                                ^^^^^^^^^^^^^^^^
  File "C:\repos\private\panel-140-social\.venv\Lib\site-packages\bokeh\core\serialization.py", line 253, in encode
    return self._encode(obj)
           ^^^^^^^^^^^^^^^^^
  File "C:\repos\private\panel-140-social\.venv\Lib\site-packages\bokeh\core\serialization.py", line 282, in _encode
    return self._encode_dict(obj)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "C:\repos\private\panel-140-social\.venv\Lib\site-packages\bokeh\core\serialization.py", line 343, in _encode_dict
    entries=[(self.encode(key), self.encode(val)) for key, val in obj.items()],
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\repos\private\panel-140-social\.venv\Lib\site-packages\bokeh\core\serialization.py", line 343, in <listcomp>
    entries=[(self.encode(key), self.encode(val)) for key, val in obj.items()],
                                ^^^^^^^^^^^^^^^^
  File "C:\repos\private\panel-140-social\.venv\Lib\site-packages\bokeh\core\serialization.py", line 253, in encode
    return self._encode(obj)
           ^^^^^^^^^^^^^^^^^
  File "C:\repos\private\panel-140-social\.venv\Lib\site-packages\bokeh\core\serialization.py", line 299, in _encode
    return self._encode_other(obj)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\repos\private\panel-140-social\.venv\Lib\site-packages\bokeh\core\serialization.py", line 477, in _encode_other
    self.error(f"can't serialize {type(obj)}")
  File "C:\repos\private\panel-140-social\.venv\Lib\site-packages\bokeh\core\serialization.py", line 480, in error
    raise SerializationError(message)
bokeh.core.serialization.SerializationError: can't serialize <class 'pydeck.types.function.Function'>

[MarcSkovMadsen]

From what I can see 2 things are needed

Function Serializer

See also https://deck.gl/docs/api-reference/json/conversion-reference.

Maybe instead of doing our own serialization of the pydeck .__dict__ it would be better to use the serialization that must be built into pydeck?

Configuration Support

PyDeck Carto adds some custom configuration of a js function that Panel needs to support. Similarly to existing support for tooltips.

https://github.com/visgl/deck.gl/blob/479ebd103c546f19006debeef8ccc8bb5c3ca37e/bindings/pydeck-carto/pydeck_carto/layer.py#L43

[MarcSkovMadsen]

I've been looking at this. I can see the Jupyter Deck.gl widget supports 3 map providers: mapbox, carto and google_maps. Furthermore it supports adding custom functions (needed by carto) and custom libraries (needed by carto). To support these 3 map providers we would need to do a similar implementation. Seee https://github.com/visgl/deck.gl/blob/master/modules/jupyter-widget/src/playground/create-deck.js.