Band math expressions for client-side rendering

[ddohler]

Hello! I enjoyed the presentation of this extension at FOSS4G NA yesterday. I wanted to follow up on the conversation about band math expressions that was started there.

My use case is that I'm styling rasters on the frontend using OpenLayers and would be interested in using this extension to store style information. The OpenLayers style expressions look like this (for NDVI): ['/', ['-', ['band', 2], ['band', 1]], ['+', ['band', 2], ['band', 1]]] . MapBox GL and MapLibre would look similar, if not exactly identical. Looking at the definition of expression for this extension, it looks like the equivalent would be "(B02–B01)/(B02+B01)". I'm not sure what library is being used to parse the formulas in Titiler, but an equivalent would need to be available in Javascript in order for someone to make use of these styling expressions on the frontend.

So that's basically the crux of it -- in order for this to be usable on the frontend, there should be a way to convert the band-math expressions into the style expressions expected by frontend mapping libraries and I'm not sure whether there is. If you can provide guidance, that would be great--thank you!

[j08lue]

The Python code parsing and applying the expressions in TiTiler / rio-tiler is here: https://github.com/cogeotiff/rio-tiler/blob/741acc3596d908db1755c82f18e2f4fd3711c49c/rio_tiler/expression.py#L80-L88

Basically, after a bit of parsing / splitting into blocks (defining output bands), the expressions are passed to numexpr.evaluate

numexpr.evaluate(bloc.strip(), local_dict=dict(zip(bands, data)))

with a mapping from bands to dimensions in the data.

But that all said - I'd say the expression format is specific to the application that is supposed to apply it, TiTiler in our case. Instead of translating them to various formats, you could add another renderer specifically for OpenLayers. 🤷

Not sure there is any standard and if it would help anyone to follow that - QGIS has raster calculator expressions, Sentinel Hub has custom scripts / Evalscript...

[hrodmn]

:wave: @ddohler - thanks for opening up this discussion! I was very focused on titiler when we discussed this at FOSS4GNA this week, but @j08lue is right - you can put whatever kind of expression syntax that you want in the expression slot. I don't think the intention of this extension is to standardize the expression syntax across all visualization engines, so it would be good to keep it flexible.

However, the spec says that the expression data type is str so maybe we would want to expand the spec to be more flexible and include json as a possible data type for that field.

@j08lue's idea for a separate renderer is a good one. I can imagine a collection with a set of renderers that could be used in different applications that would all yield consistent results.

e.g. NDVI from Landsat for titiler and OpenLayers:

{
  "renders": {
    "ndvi-titiler": {
      "title": "Normalized Difference Vegetation Index",
      "assets": ["B04", "B05"],
      "expression": "(B05 – B04) / (B05 + B04)",
      "resampling": "average",
      "colormap_name": "ylgn"
    },
    "ndvi-openlayers": {
      "title": "Normalized Difference Vegetation Index",
      "assets": ["B04", "B05"],
      "expression": [
        "/",
        ["-", ["band", 2], ["band", 1]],
        ["+", ["band", 2], ["band", 1]],
      ],
      "resampling": "average",
      "colormap_name": "ylgn"
    }
  },
}

I have not used OpenLayers so I don't know what the colormap parameters would actually be, but you would do whatever you need to do to have a common visualization output from each application.

[hanbyul-here]

Thanks for the follow-up 🙇 It is such a great idea to centralize the style expression. As @hrodmn detailed, the type of the expression field can be expanded. but I also wonder if it would make more sense to have your own additional property for open layer rather than trying to use expression?

[ddohler]

Thanks all! It sounds like either of these two options (expand the type of the expression field or add additional fields for other expression formats) could work.

Just to clarify, @hanbyul-here what I think you are suggesting is something like this, is this correct?

{
  "renders": {
    "ndvi": {
      "title": "Normalized Difference Vegetation Index",
      "assets": ["B04", "B05"],
      "expression": "(B05 – B04) / (B05 + B04)",
      "expression-openlayers": [
        "/",
        ["-", ["band", 2], ["band", 1]],
        ["+", ["band", 2], ["band", 1]],
      ],
      "resampling": "average",
      "colormap_name": "ylgn"
    },
  },
}

The advantages I see of this approach are that it doesn't require repeating the title, assets and other keys, and it also provides guidance about what each expression is for. The main disadvantage I see is that it means any renderer that is not able to support the numexpr strings will be forced to define a custom field, which seems like it would reduce the usefulness of standardization, if many renderers start relying on custom non-defined fields.

I think between the two I would probably prefer expanding the allowed types for expression (perhaps as string | Object | Array?) and writing multiple entries in renders, because I could also see the colormaps needing to be different; clientside rendering will probably need to use the expanded colormap object syntax rather than simply passing in a name.

But that's only a slight preference; I think either option would work. If y'all choose to keep expression titiler-specific then I think it would be good to make sure that the option to add a renderer-specific custom expression-* field is clearly documented, with perhaps some suggestions for field names to use for popular libraries.

Thanks for the good discussion, I'm cool with either path!

[smohiudd]

Thanks all for starting this discussion. As @j08lue and @hrodmn mentioned already the intent of render is to be tiler or front end agnostic. I don't think the schema should be so locked down that it can't be used for multiple expression types. My vote is to broaden the allowed types for expression.

[smohiudd]

PR https://github.com/stac-extensions/render/pull/9