Expression engine extension points

[nreese]

Motivation

I work on a mapping application that allows users to configure formatters to customize how values are displayed. These formatters are javascript functions that take a value and return a string. I would like to be able to use these formatters in a maplibre expression to format values displayed as labels.

Below are 2 images. The first uses 'vector' source and displays the raw value. The second uses 'geojson' source and adds a new property to each feature that contains the value transformed into a human readable byte string. It is not possible to display formatted values with 'vector' source. Allowing expression extension points would close this feature gap and allow custom label formatting with 'vector' source.

Here are some other example of formatting functions

• Relative timestamp. for example: "12 days ago", "5 minutes ago", etc
• durations, for example: "2 hours", "5 minutes", etc
• percentages
• static look-up tables
• short dots transformation, for example: "com.organizations.project.ClassName" becomes "c.o.p.ClassName"

Beyond these, there are limitless possibilities for formatting values. Custom expressions provides an escape hatch to allow users easily fill any functionality gaps.

There are many additional use cases and further discussion can be found at https://github.com/mapbox/mapbox-gl-js/issues/9462

Design Alternatives

Custom expression logic could be avoided by performing the logic at vector tile generation. However, this is not always possible. In the use case above, the vector tiles are generated from the data store and logic for formatting labels is not available at the data store abstraction level.

Users could also fall back to 'geojson' sources and add new properties to features as a work around. However, falling back to 'geojson' sources provides a much slower alternative when large data volumes are involved.

Design

Mock-Up

The below mock-up is copied from https://github.com/mapbox/mapbox-gl-js/issues/9462

This is modelled after how expression functions are defined in definitions/index.js.

Note that its using the type names defined in compound_expression.js

map.registerExpression(
  'my-function', // Function name
  StringType, // Type
  [ValueType], // Signature
  (evaluationContext, args) => {
     ...
  } // Evaluate
);

[HarelM]

Can you share the style that you used for the example pictures above? The main problem I see here is that you can't describe it in a style file. One of the good things about the style is that it is declarative. Having said that, it's not a must, just like addProtocol is breaking the declarative nature of the style. My only question here is elaborating on the use cases as I tend to say that in most cases this can be solved by the current code...?

[nyurik]

Assuming this is really needed (judging by the upstream issue https://github.com/mapbox/mapbox-gl-js/issues/9462), there are several paths to implement this:

• allow users to register a custom expression, expanding the list of the built-in functions with custom functions that accept arbitrary params. The biggest drawback is a requirement for API stabilization -- we will have to document and maintain registration and the callback API specs, which will make it a bit harder to make internal changes. This will also not be portable to native/rs.
• introduce a new text-custom-transform expression similar to text-transform, so it would have far smaller API to maintain. It would simply take the name of the registered custom transformation, a text string, and any extra parameters from the style (params could be computed on the fly by MapLibre if needed). Usage: "text-custom-transform": "to-decimal" or "text-custom-transform": ["to-decimal", "arg1", "arg2", ...] -- where to-decimal is a registered function that accepts the content of the text-field value and can accept any optional arguments.
• allow users to override feature data accessor, i.e. when MapLibre reads tile data after decoding it, be able to dynamically change that data before it gets processed. I suspect this won't be very performant, esp because the callback would have to know a lot of context (layer ID, other tags, etc) and evaluate all that on each feature.
  • a more specific override could be introduced on a specific field, e.g. only capture access to the ["get", "name_en"] and ignore all else.

[nreese]

Can you share the style that you used for the example pictures above?

vector source style

``` { "version": 8, "glyphs": "https://tiles.maps.elastic.co/fonts/{fontstack}/{range}.pbf", "sources": { "78ba9b28-7b24-4691-afad-57bed27d2d33": { "type": "vector", "tiles": [ "/irk/api/maps/mvt/getTile/{z}/{x}/{y}.pbf?geometryFieldName=geo.coordinates&index=kibana_sample_data_logs&hasLabels=true&requestBody=(_source%3A!f%2Cdocvalue_fields%3A!(bytes)%2Cquery%3A(bool%3A(filter%3A!((range%3A(timestamp%3A(format%3Astrict_date_optional_time%2Cgte%3A'2022-05-27T19%3A40%3A39.741Z'%2Clte%3A'2022-06-03T19%3A40%3A39.741Z'))))%2Cmust%3A!()%2Cmust_not%3A!()%2Cshould%3A!()))%2Cruntime_mappings%3A(hour_of_day%3A(script%3A(source%3A'emit(doc%5B!'timestamp!'%5D.value.getHour())%3B')%2Ctype%3Along))%2Cscript_fields%3A()%2Csize%3A10000%2Cstored_fields%3A!(bytes%2Cgeo.coordinates))&token=c992830b-9c90-49d1-9aa4-87fe57b07866" ], "minzoom": 0, "maxzoom": 24 } }, "layers": [ { "id": "78ba9b28-7b24-4691-afad-57bed27d2d33_circle", "type": "circle", "source": "78ba9b28-7b24-4691-afad-57bed27d2d33", "source-layer": "hits", "minzoom": 0, "maxzoom": 24, "filter": [ "all", [ "!=", [ "get", "_mvt_label_position" ], true ], [ "any", [ "==", [ "geometry-type" ], "Point" ], [ "==", [ "geometry-type" ], "MultiPoint" ] ] ], "layout": { "visibility": "visible" }, "paint": { "circle-color": "#54B399", "circle-opacity": 0.75, "circle-stroke-color": "#41937c", "circle-stroke-opacity": 0.75, "circle-stroke-width": 1, "circle-radius": 6 } }, { "id": "78ba9b28-7b24-4691-afad-57bed27d2d33_label", "type": "symbol", "source": "78ba9b28-7b24-4691-afad-57bed27d2d33", "source-layer": "hits", "minzoom": 0, "maxzoom": 24, "filter": [ "all", [ "==", [ "get", "_mvt_label_position" ], true ] ], "layout": { "text-field": [ "coalesce", [ "get", "bytes" ], "" ], "text-size": 14, "visibility": "visible" }, "paint": { "text-color": "#000000", "text-opacity": 0.75, "text-halo-width": 1, "text-halo-color": "#FFFFFF" } }, ] } ```

geojson source style

``` { "version": 8, "glyphs": "https://tiles.maps.elastic.co/fonts/{fontstack}/{range}.pbf", "sources": { "78ba9b28-7b24-4691-afad-57bed27d2d33": { "type": "geojson", "data": { "type": "FeatureCollection", "features": [ { "type": "Feature", "geometry": { "coordinates": [ -143.5770444, 70.13390278 ], "type": "Point" }, "properties": { "__kbn__feature_id__": "kibana_sample_data_logs:rPg6JIEBVqWP38Dm1ah7:0", "bytes": 2533, "timestamp": 1654264627624, "_id": "rPg6JIEBVqWP38Dm1ah7", "_index": "kibana_sample_data_logs", "__kbn__dynamic__bytes__labelText": "2.5KB" }, "id": 3228 }, { "type": "Feature", "geometry": { "coordinates": [ -156.7660019, 71.2854475 ], "type": "Point" }, "properties": { "__kbn__feature_id__": "kibana_sample_data_logs:h_g6JIEBVqWP38Dm1ah7:0", "bytes": 6226, "timestamp": 1654253705113, "_id": "h_g6JIEBVqWP38Dm1ah7", "_index": "kibana_sample_data_logs", "__kbn__dynamic__bytes__labelText": "6.1KB" }, "id": 3230 }, { "type": "Feature", "geometry": { "coordinates": [ -148.4651608, 70.19475583 ], "type": "Point" }, "properties": { "__kbn__feature_id__": "kibana_sample_data_logs:Lvg6JIEBVqWP38Dm1Kf3:0", "bytes": 8503, "timestamp": 1654062488489, "_id": "Lvg6JIEBVqWP38Dm1Kf3", "_index": "kibana_sample_data_logs", "__kbn__dynamic__bytes__labelText": "8.3KB" }, "id": 3229 }, { "type": "Feature", "geometry": { "coordinates": [ -151.0055611, 70.20995278 ], "type": "Point" }, "properties": { "__kbn__feature_id__": "kibana_sample_data_logs:sPg6JIEBVqWP38Dm1KX3:0", "bytes": 3682, "timestamp": 1653926802851, "_id": "sPg6JIEBVqWP38Dm1KX3", "_index": "kibana_sample_data_logs", "__kbn__dynamic__bytes__labelText": "3.6KB" }, "id": 3231 } ] } } }, "layers": [ { "id": "78ba9b28-7b24-4691-afad-57bed27d2d33_circle", "type": "circle", "source": "78ba9b28-7b24-4691-afad-57bed27d2d33", "minzoom": 0, "maxzoom": 24, "filter": [ "all", [ "!=", [ "get", "__kbn_is_centroid_feature__" ], true ], [ "any", [ "==", [ "geometry-type" ], "Point" ], [ "==", [ "geometry-type" ], "MultiPoint" ] ] ], "layout": { "visibility": "visible" }, "paint": { "circle-color": "#54B399", "circle-opacity": 0.75, "circle-stroke-color": "#41937c", "circle-stroke-opacity": 0.75, "circle-stroke-width": 1, "circle-radius": 6 } }, { "id": "78ba9b28-7b24-4691-afad-57bed27d2d33_label", "type": "symbol", "source": "78ba9b28-7b24-4691-afad-57bed27d2d33", "minzoom": 0, "maxzoom": 24, "filter": [ "all", [ "any", [ "==", [ "geometry-type" ], "Point" ], [ "==", [ "geometry-type" ], "MultiPoint" ] ] ], "layout": { "text-field": [ "coalesce", [ "get", "__kbn__dynamic__bytes__labelText" ], "" ], "text-size": 14, "visibility": "visible" }, "paint": { "text-color": "#000000", "text-opacity": 0.75, "text-halo-width": 1, "text-halo-color": "#FFFFFF" } }, ] } ```

[nreese]

I would like to be able to register an expression extension so instead of

"layout": {
  "text-field": [
    // text-field value is computed:   coalesce(get("bytes"), "")
    "coalesce", [ "get", "bytes" ], ""
  ],
  "text-size": 14
}

I could do something like the below, where kibana-text-transform is an extension I have registered with maplibre and it takes a value, runs a callback function and returns a value.

"layout": {
  "text-field": [
    // text-field value is computed:   my_custom(coalesce(get("bytes"), ""))
    "my_custom": [
      "coalesce", [ "get", "bytes" ], ""
    ],
  ],
  "text-size": 14
}

[nyurik]

@nreese thx for the example. I agree custom expression functions would be the most flexible solution. It might also put too much maintenance burden compared to a dedicated text transformation function. Compare the API requirement for an expression extension that can do anything (i.e. has execution context and supports all data types as inputs and outputs), vs a text-only function with no context. I will let @HarelM speak about the API complexities/burderns, as I might be totally wrong here :)

My usage example with a custom text function:

"layout": {
  "text-field": [
    // text_custom_transform("my_custom", coalesce(get("bytes"), ""))
    // note that the second value could be "{name_en}" instead of an array
    "text-custom-transform": [
      "my-custom",
      ["coalesce", [ "get", "bytes" ], ""],
    ]
  ],
  "text-size": 14
}

[wipfli]

There is this concept of runtime styling versus the style.json approach. I don't quite understand it but maybe we should be careful to not mix the two. Can someone comment on the declarative thing? Sorry if I am a bit confused...

[wipfli]

The point is that if we introduce this feature, the style file alone will not tell you how the map looks like. You need style plus callback code. This is not necessarily bad, but we have to have a plan how this would look like on the native project...

[wipfli]

@nreese can you maybe share some more use case examples? I tend to say that we should rather expand the expressions than introduce a more general callback registration functionality. The cool thing about the current way of things is that you can copy-paste style sections. This works because they are sort of self-describing.

For your example above. Turning 2536 into 2.5KB can be achieved with something like this: ['concat', ['to-string', ['/', ['ceil', ['/', 2536, 100]], 10]], 'KB']

[wipfli]

It is not possible to display formatted values with 'vector' source.

I did not know that. Always thought that you can do something like ['get', 'name_en'] on a vector source. Or maybe I am misunderstanding?

[nyurik]

Per @1ec5 there was also a relevant discussion on the native side in https://github.com/mapbox/mapbox-gl-native/issues/7860

[nreese]

can you maybe share some more use case examples

For my specific use case in Kibana, there are many formatters so it would not be trivial to solve them all with expression code. There are durations, percentages, static look-up tables, and more.

For the specific example of displaying a value as bytes, ['concat', ['to-string', ['/', ['ceil', ['/', 2536, 100]], 10]], 'KB'] would not work as values need to be expressed as MB or GB or whatever suffix depending on the size. Coding all of this in an expression is non-trivial and error prone.

[nreese]

I did not know that. Always thought that you can do something like ['get', 'name_en'] on a vector source. Or maybe I am misunderstanding?

@wipfli This was addressed as a "Design Alternatives"

"Custom expression logic could be avoided by performing the logic at vector tile generation. However, this is not always possible. In the use case above, the vector tiles are generated from the data store and logic for formatting labels is not available at the data store abstraction level."

The short answer being, it is not always possible to add formatted data to the tile generation process.

[wipfli]

In yesterday's technical steering committee meeting, the following came up: Americana's creative usage of the missing style image was mentioned by @mojodna as a way to get run-time callbacks from the style.

[1ec5]

Americana's creative usage of the missing style image was mentioned by @mojodna as a way to get run-time callbacks from the style.

The style image callback runs well after style evaluation: https://github.com/ZeLonewolf/openstreetmap-americana/pull/243#issuecomment-1104269627. By contrast, anything done as part of evaluating an expression has to run in a Web worker (multiple Web workers), which complicates things a bit. Some languages like Swift make it possible to prevent unintended capturing of variables outside of scope, but I don’t know if that’s the case for TypeScript.

[nyurik]

Seems like this got a bit stalled. Another use case where I think we do not have a good answer is a mix of static and dynamic data. For example, let's say there are millions of data points with metadata. Each point can be styled based to that metadata, and have some popup which shows that metadata on a click/mouseover. But now let's say there is some dynamic "current state" -- something that rapidly changes each second. I could subscribe a stream of (<feature ID>, <status>) feed that updates some data storage in memory, and I could animate those points on the screen, e.g. show the point as green or red.

Ideally, I would love to just add a styling rule like get_dynamic_state(feature_id) that executes my own function, which performs a lookup in the dynamic state and tells the styling system which color to use with the current feature.

[HarelM]

Doesn't setFeatureState solves this?

[nyurik]

@HarelM sorry for not replying earlier - yes, the feature state concept fully solves it, thanks!

[cyrilchapon]

Hi. Joining the conversation as I'm facing a use-case related to this feature. Documented here #2077

Basically, I'm trying to color some symbol icons based on the distance relative to a GeoJSON point.

The distance operator defined in the spec would be a perfect fit. Unfortunately, neither mapbox or maplibre implements this in gl.js There was a great draft PR which basically implements it perfectly in mapbox; but it has been blocked in draft for almost 2 years because of a significant bundle size increase.

As stated in #2077; "significant bundle size increase" + "already implemented" sounds for a perfect "plugin" fit.

---

I think this is the perfect use-case because its both :

• Totally compatible with static-style (geometry comparator is just a GeoJSON point object)
• Relatively easy to implement as a logic
• Hardcore to integrate inside the lib

In other words, with an expression "extension" mecanism; this would be roughly implementable in 1 hour — and would even be style-spec compliant; but in the current state of art this is very tough to implement.

[DevelWolf]

Moved from: https://github.com/maplibre/maplibre-gl-js/issues/2077#issuecomment-1478551536

[DevelWolf]

I'm not sure how easy it will be to add an expression as a plugin, but if you manage to do it, or even create a PR that will allow to add expression as plugins it can open a world of opportunities.

To understand what's needed for a "distance"-plugin, I have patched a rudimentary implementation of distance expression, which only computes the distance between point features, into the current version of maplibre-gl.js. You can try it here.

As you can see, the implementation of distance needs access to the EvaluationContext ctx, more precisely the members geometryType() and geometry() to get the feature's pixel coordinates, and to member canonical of type ICanonicalTileID to convert the pixel coordinates to LonLat.

It would be quite easy to naively add a hook Map.addExpressionType(...) to dynamically add expression types to the library, but this would need to expose the EvaluationContext and member types, which I think is a bad idea.

Considerations for a clean implementation

1. Encapsulation of context access

There already is a geometry-type expression (spec), which delivers the ctx.feature.type as string.

We can easily add a expression geometry to the library, which returns the feature's geometry as an object, already converted from pixel coordinates to geographic coordinates.

2. Providing a hook for adding native expression types

With "native" I mean the function will receive evaluated values of JS base type as arguments and has no access to the enviroment or other parts of the library at all.

Map.addNativeExpressionType( 'mean',
        'number', ['number', 'number'],
    (a, b) => return (a+b) / 2
);

Map.addNativeExpressionType( 'string-index-of',
    'number', ['string', 'string'],
    (needle, haystack) => return haystack.indexOf(needle)
);

Map.addNativeExpressionType( 'string-char',
    'string', ['string', 'number'],
    (s, index) => return s.charAt(index)
);

This way we can implement the distance computation as a native expression type:

Map.addNativeExpressionType( 'explicit-distance',
    'number', ['string', 'object', object],
    (geoType, geo, referenceFeature) => {
        //some math
        return distance;
    }
);

and use it:

['explicit-distance',
    ['geometry-type'],
    ['geometry'],
    {
        type: 'Feature',
        geometry: {
            type: 'Point',
            coordinates: [2, 40]
        }
    }
]

3. Providing a hook for adding "macro expression" types

Unfortunately this way we cannot provide the distance expression type exactly as specified in the documentation, because the distance needs implicit access to the context.

To fix this, we could add "macro expressions", which would not be added to the list of available expressions, but used while compiling an expression; may be this way (I'm not sure about the syntax):

Map.addMacroExpressionType( 'distance', 1 /*one arg*/,
    [ 'explicit-distance', ['geometry-type'], ['geometry'], [1/*placeholder for arg 1*/] ]

Resulting in the expression

['distance',
    {
        type: 'Feature',
        geometry: {
            type: 'Point',
            coordinates: [2, 40]
        }
    }   
]

being replaced by and compiled as

['explicit-distance',
    ['geometry-type'],
    ['geometry'],
    {
        type: 'Feature',
        geometry: {
            type: 'Point',
            coordinates: [2, 40]
        }
    }
]

Heureka.

Addendum: the literal object {type: ...} has to be written as ['literal', {type: ...}]

[HarelM]

Can you better clarify the difference between the two APIs? From my point of view there should be a single API registerCustomExpression or something similar. Why is gaining access to the evaluation context is problematic?

[DevelWolf]

access to the evaluation context

Why is gaining access to the evaluation context is problematic?

Because you have to use the MapLibre library source while creating the plugin and use version numbers to verify the compatibility between a plugin and the library. Which is cumbersome and overkill if the only thing you want is to define a ['replace-all'].

Otherwise, you would have to freeze the EvaluationContext class, which is something I certainly do not want.

register native expressions

Can you better clarify the difference between the two APIs?

The calls have nothing in common and have completely different signatures. In fact, only the first one is important for the very registration of custom expression types.

register macro expressions

I made up the second one only to allow implementing the distance expression exactly as specified in the documentation without having to give native custom expressions access to library elements not exposed to the public.

Currently, I see no other usage. In fact, if it were only for me, I would permanently add the distance expression to the library with a rudimentary implementation, which you could replace by registering a custom distance-implementation.

not easy

Unfortunately, I have to take back the “easily”.

The expressions are evaluated both in the UI process and in the Worker processes; for this purpose, the buildin expressions are marshalled and included in the Worker blob-uri. If after creating the Worker you want to add an expression, you have to both store it in the UI process and send a serialization of it as a message to the workers, where a message handler would add it to the buildin expressions.

For the same reason, custom expression also have no access to static variables, i.e. lookup tables; the possibilities are therefore very limited. Maybe they are not worth the effort at all.

crude implementation

Nevertheless, to have a basis for further discussions, I've patched the current maplibre-gl.js to allow custom expressions as maplibre-gl+distance+custom-expressions-v1.js. It does not use the Worker message system, but I have set the library initialization on hold, and you have to explicitly initialize the library after you have defined your custom expressions.

You can find a usage example as jsfiddle.

This hack is for obvious reasons not intended as a base for a real implementation.

[HarelM]

I agree a plugin system such as one that is discussed here should have a good interface that would change rarely, like the Cordova/Ionic plugin interface/system, which holds nicely for a long time now. In theory one could wrap the context in another class to facilitate for the relevant plugin capabilities. I like a general registerExpression method to facilitate for as many options as possible. I'll be happy to review a design if someone would like to push this forward.

[DevelWolf]

The current implementation of CompoundExpression.register() in src/expression/compound_expression.ts allows it to be called only once; a second call would destroy the former registration.

I suggest a small (obvious) patch, which makes the method callable multiple time.

With this change, you get the internal hook to design an interface around.

***************
*** 31,37 ****
      _evaluate: Evaluate;
      args: Array<Expression>;

!     static definitions: {[_: string]: Definition};

      constructor(name: string, type: Type, evaluate: Evaluate, args: Array<Expression>) {
          this.name = name;
--- 31,37 ----
      _evaluate: Evaluate;
      args: Array<Expression>;

!     static definitions: {[_: string]: Definition} = {};

      constructor(name: string, type: Type, evaluate: Evaluate, args: Array<Expression>) {
          this.name = name;
***************
*** 146,153 ****
          registry: ExpressionRegistry,
          definitions: {[_: string]: Definition}
      ) {
-         CompoundExpression.definitions = definitions;
          for (const name in definitions) {
              registry[name] = CompoundExpression;
          }
      }
--- 146,153 ----
          registry: ExpressionRegistry,
          definitions: {[_: string]: Definition}
      ) {
          for (const name in definitions) {
+             CompoundExpression.definitions[name] = definitions[name];
              registry[name] = CompoundExpression;
          }
      }