[Question] - Steps to create new component

[sweco-semtne]

Hi! I'm not sure where to direct my question so apologies if this isn't the correct forum or place.

I'm trying to add a dynamic map component to Kino to interact with geographical data. It is loosely based on Folium

I'm trying to understand the flow and extension points in Kino and Livebook. My understanding so far:

Kino

• add a new type and mapping in Kino.Output

• add a defimpl in Kino.Render (to_livebook)

• implement a server in Kino

Livebook

• introduce a LiveComponent in Livebook for rendering the output

• create a hook for the client side JS part of it

• add a type in Livebook.Notebook.Cell.Elixir

Are there any other points to consider?

Also I would like to thank all of your building such a fantastic tool! Keep up the good work!

[jonatanklosko]

Hey @sweco-semtne, the points you listed are accurate. There are two points sides to components.

The first one is the output specification (currently documented in Kino.Output) - these are the recognised formats and on Livebook side this translates to a piece of UI for the given type.

The other side is rendering the given Elixir term into that specific output (implementing Kino.Render), in most cases we have dedicated Kino.* structs.

The output format may be more generic, for example we have the :table_dynamic format (rendered as a table in Livebook), but we have several components using this: Kino.ETS, Kino.DataTable, Kino.Ecto. A server process (on Kino side) is only necessary for components that need to dynamically exchange data with Livebook, in other cases we simply convert structs to static output like {:vega_lite, %{plot definition here}}.

As for the UI, it depends on the case, we currently use JS only for rendering Vega-Lite plots (and that does require a hook).

[1/2]

[jonatanklosko]

[2/2]

We are definitely open to adding new components, but in cases where it requires integrating an external JS package we need to be careful, because every such package may significantly increase the bundle size. What's the exact idea behind the map component? What kind of Elixir data would that render? Also, see a relevant question here https://github.com/livebook-dev/livebook/issues/305 :)

[jonatanklosko]

Looking at Folium, I can see that they manipulate the map using leaflet.js, so based on the Python code they generate some JavaScript from a template and then let this JavaScript run. So to do something similar we would need either:

• let the output be arbitrary HTML/JS, which we don't want given the security risks
• have a lot of custom JS on Livebook side, or rather create a package that takes a JSON specification and renders the map; however it would need to be something standardized like Vega-Lite

[sweco-semtne]

@jonatanklosko thank you for a very thorough answer. I was thinking that the output would basically be wrapping the concept of a data table in the sense that it could take a table as input along with a definition of which columns contains the geographical coordinates.

Or a data frame similar to GeoPandas.

The reason for having a Server is to be able to push live data on to the map canvas for for instance GPS positions.

It could also possible be used to filter tables on geography if a map input component is built. That would need additional libraries like turf to do the spatial filtering.

I'm not looking to use leaflet but instead the MapLibre library which is a fork of Mapbox 1.x

The library has the concept of data sources that can be updated with a setData function that would map well to hook.

Under the hood the input data could be converted to GeoJSON features.

I agree that some kind of specification on what background maps, styles etc to use should be formalized but I'm not sure there is any standard worth sticking to. There is the old OGC Web Map Context and several other proprietary means of serializing the map context.

I'll try to come up with an example flow to better describe what I'm thinking.

Again, thank you for taking the time to respond and also investigate!

[josevalim]

After some discussions with @jonatanklosko and @thbar yesterday, we think it is work adding some support for map rendering in Livebook/Kino. We mentioned VegaLite before but it seems Vega does not have support for tiling yet.

My initial suggestion is to support GeoJSON rendering exclusively, so the API would rather evolve around GeoJSON, with some basic configuration for picking the tile provider. Would that be helpful to you @thbar and @sweco-semtne?

Also, does anyone have a rundown on the differences or solid preferences towards MapLibre or Leaflet?

[sweco-semtne]

Hi @josevalim That's great news. GeoJSON as data format is a good middleground. I've worked with both OpenLayers, Leaflet and MapLibre(MapBox 1.0) and for me personally I like the data source + multiple layer abstraction in MapLibre. I would say that Leaflet and OL has a richer set of plugins. But for this use case I would go with MapLibre. Or maybe take a look at vis.gl and the tools around luma?

On a side note, we've been using Liveview with Broadway to push thousands of vehicle position updates to a Mapbox/MapLibre map without any performance issues. So thank you very much for all of these fantastic libraries/frameworks!

[sweco-semtne]

The Style specification for MapLibre can be used as a basis for a map "context".

The GeoJSON specifies the data but there still is a need to specify how to visualize it, ie layer order, colors, shapes etc

https://maplibre.org/maplibre-gl-js-docs/style-spec/

[thbar]

My initial suggestion is to support GeoJSON rendering exclusively, so the API would rather evolve around GeoJSON, with some basic configuration for picking the tile provider. Would that be helpful to you(...) ?

I believe a GeoJSON layer + tile background rendering would be a natural first step, and would probably take us quite far already. It would definitely be useful to the various data explorations we do at https://transport.data.gouv.fr.

Let me share extra ideas that could be useful for later (these are things we are also looking at) ; these are currently overkill, but keeping them as options in mind could help design something that can go further later:

• Support code generating GeoJSON on the fly, based on the currently viewed area (bounding box), so that we can have more dynamic scenarios in the future (clustering of large data, moving around etc)
• Support for MVT (MapBox Vector Tiles), which are protobuf-compressed dynamically generated data (for the main data, not the background) ; this supports much larger datasets, and I am mentioning this because we see more large-data stuff coming out recently (in the spirit of Explorer, Nx/Axon etc). I have not found native support for this in Elixir yet, but will most likely dive into it (interesting link: https://github.com/urbica/martin)

Also, does anyone have a rundown on the differences or solid preferences towards MapLibre or Leaflet?

I am not familiar at all with MapLibre. At https://transport.data.gouv.fr, we currently use LeafLet and mostly quite happy with it.

I'll add a couple of interesting links to the discussion:

• https://github.com/leaflet-extras/leaflet-providers -> having different tiling servers support will be helpful
• http://leaflet-extras.github.io/leaflet-providers/preview/index.html -> demo of the above
• https://github.com/topojson/topojson -> I have seen this used as layer with VegaLite, a middle-ground of optimization

[sweco-semtne]

@thbar MapLibre is the community fork of Mapbox pre 2.0. So it supports vector tiles out of the box. Or did you mean the library should support generating it in Elixir-land? I was also thinking about having a map input. Is that what you mean with the bounding box?

And it would be interesting to wrap a data table with a function to transform it to GeoJSON.

[thbar]

MapLibre is the community fork of Mapbox pre 2.0. So it supports vector tiles out of the box.

Neat, thanks for clarifying this!

Or did you mean the library should support generating it in Elixir-land?

This is a great question! Yes, I meant "Support /Elixir/ code generating GeoJSON on the fly", definitely. Having "server-side" (Elixir-land) data generation will allow to process large amounts of data, cluster them if needed, clip as needed, and only send what is relevant to the rendering part.

Is that what you mean with the bounding box?

By bounding box I meant: the end user could in the future be able to control the map with arrows, and the resulting viewport (lat/lon top left and bottom right) gives coordinates, which can be "sent back" to the Elixir part, so that it only generates the right bits of data.

If needed, I'll emphasise that a first version with no clipping of any sort, "just" rendering an Elixir-generated GeoJSON on a map, and letting the user configure a tiling background where the coordinates would automatically use the same referential and be well aligned with the GeoJSON, would already provide a huge value (the bounding box idea is more fuel for thoughts for later in my head!).

[sweco-semtne]

I'll just add one more link for inspiration: https://geopandas.org/docs/user_guide/interactive_mapping.html

The GeoPandas library just introduced the explore function that creates an interactive leaflet map based on a dataframe.

[bt-maps]

Very interested in this conversation as I work in the web mapping / data science field where I use and develop mapping components regularly. New to Elixir and loving livebook/kino so I would like to add deck.gl to the conversation. deck.gl supports a declarative system for describing layers and their props, and this declarative API can be trivially exposed via the JSON API. Hoping this may make it easier to integrate.

My opinion is a webgl-based mapping library (maplibre, mapbox v1, or deck.gl) would improve livebook more than a leaflet-based solution.

see bindings for python (pydeck), Dash Deck , R (mapdeck), Vega(vega-deck.gl) at https://github.com/visgl/deck.gl

Brent

[roger120981]

Currently in elixir there are no libraries equivalent to Geopandas, which in its internal development makes use of other mature and stable projects such as shapely to work with geometries, pandas for DataFrame, etc.

The first thing would be to create a similar project in elixir that will enhance the use of the platform in this niche. There are two real options:

1- Being Rust a platform that has projects similar to python in this niche, create elixir NIFs with Rustler that brings functionalities to the elixir ecosystem and use them in a transparent way. Creating similar projects from scratch would be very expensive.

2- Find an efficient way to integrate python into livebook and call these functionalities without much cost to the system

Later it would be used to display Deck.gl or Maplibrejs, due to its webgl rendering capabilities.

[bt-maps]

Being new to Elixir, I have been spending some time getting up-to-speed with the Elixir geospatial libraries. Wow! there are some really good libraries available to build the core of a geospatial data handling/indexing solution. geo - Encodes and decodes WKB, WKT, and GeoJSON formats. Will also handle standard geometry formats from databases (PostGIS, MySQL) topo - Geometry library for determining spatial relationships between geometries envelope - A library for calculating envelopes of geometries and tools to compare them. spatial_map - Geospatial feature storage for fast intersection checks

Seems like pkinney would be someone that might have some good ideas into this conversation since he developed quite a few of these libraries that cover core geospatial data formats and storage/indexing requirements.

So we already have the libraries above to use within livebook. The missing part is still the map feature and tile rendering - A package that takes a JSON specification and renders the map like @jonatanklosko mentioned above seems like a really good solution when you take into account the existing elixir geo libraries.

Brent

[josevalim]

Yes, after a reevaluation, I am thinking that maplibre plus built-in support for the geo types/encodings may be the way to go. deck.gl looks very nice but my understanding is that it builds on top of of a map library, so we have to start with a map library first.

[bt-maps]

Maplibre.js and geo type support would be amazing. This is really great news.

[bt-maps]

Also just came across this component which is the beginnings of a simple SVG map renderer that handles OSM map tiles. https://hex.pm/packages/live_map

[josevalim]

Hi everyone! We have pushed an initial implementation of Kino.JS, which allows anyone to create a Kino widget! So go ahead with your wildest ideas! :D You can see #64 as an example.