Helper for appending or prepending to data layer arrays

[kennethbruskiewicz]

In the spirit of #241, I want to propose a helper function that can help beat the curse of layouts in limited cases.

Sometimes when I edit layouts, I am not doing more than appending or prepending to a part of the list in a data layer. The pattern that comes out is that I need to reference the data_layer, override the field, then reference the data_layer again just so I can get all of its existing fields.

This could probably be done in a helper function.

Cases:

fields: [
      // adding back other items
      ...LocusZoom.Layouts.get(
              "data_layer",
              "association_pvalues_catalog",
              { unnamespaced: true }
      ).fields,
      // creating new (unnamespaced) fields
      `{{namespace[assoc]}}position`,
      `{{namespace[assoc]}}pValue`,
      `{{namespace[assoc]}}consequence`,
      `{{namespace[assoc]}}nearest`,
      `{{namespace[assoc]}}beta`,
]

color: [
    // declarative rule for what should happen in matching state
    {
        field: "lz_highlight_match", // Special field name whose presence triggers custom rendering
        scale_function: "if",
        parameters: {
            field_value: true,
            then: "#FF00FF",
        },
    },
    // remember other color rules
    ...LocusZoom.Layouts.get(
        "data_layer",
        "association_pvalues_catalog",
        { unnamespaced: true }
    ).color,
]

The desideratum for the helper function appear to be:

• Can append any amount of data to the end of the property list
• Can prepend any amount of data to the beginning of the property list (for when order of list evaluation matters).
• Can compose with a means of accessing specific data_layers from a layout for editing in this way.
• When it runs, it preserves all of the previous data in their unamespaced form (which will play well with the function in #241 as long as developers only run that afterwards).

This would be easy enough to write myself (I think), but I want to be aware if there are any caveats before I do.

(I am aware that the first case listed above is eliminated by the function in #241, but there are still many places where this kind of editing would hold.)

[abought]

A more general version of this PR would be to support layout mutations in general- not just arrays.

Below are some notes and thoughts on requirements for a generic helper, along with roadblocks or unresolved questions. I may not have immediate time available to work on this but it's something I'd really like to see long term.

This function would be useful for the T2D Portal, as well as other sites (like FIVEx) that make extensive use of layout mutations to enable routine interactive functionality. It could also be used internally by widgets that perform layout mutations on behalf of the user (display_options, filter_field, etc).

Requirements / test cases

1. Provide a forwards-compatible syntax that allows users to address nested keys/values, without hard-coding assumptions about the structure of the parent layout (example: if someone is customizing a particular data layer in the associations panel, they should be able to ask for "associations data layer", not "layers[2]". The syntax should provide readable, semantic meaning).
2. Allow the matching syntax to affect one or many matches: "change the y-axis on all association plots, not just one"
3. Work on generic aspects of layouts, instead of "only plots" or "only data layers": eg, "change the point color options for stable_choice scale function within a data layer"
4. If possible, allow the user to query parts of the layout without having to learn a poorly-documented auxiliary language (a la webpack-chain)
5. Implementation should not rely on, eg, eval of arbitrary filter expressions- this will trigger CSP rules in many LZ sites. A full robust language is not needed but the syntax should be compact and expressive.

Thoughts on implementation

• To avoid hard-coded references like array[2], one option would be introduce selectors like JSONPath, which supports predicate/query/filter expressions.
  • Disadvantage: this is not a core web standard; existing implementations are pretty large libraries; can we strip this down to the barest essentials? Their predicate language is a bit too arbitrary, bordering on an eval security risk.
  • Analogy: XPath supported very clean query syntax for nested hierarchy + query predicates, like "only modify data layers of a particular type". XML was awful to write, but querying could be quite pretty.
  • Other functions exist (like lodash.get) but they don't seem to allow a generic query syntax like panels/data_layers[tag=association]
• Since layouts are essentially POJOs, we can add a new tag field to data layer layouts to reflect semantic meaning: "tag: 'association' would allow users to directly query for data layers showing association data

Proposed sample syntax LocusZoom.Layouts.mutate_attr(layout, selector_string, value_or_callable)

The selector string should be able to address one match, or many. (get me association layer for panel 1, or get me all association layers on a stacked plot)

The replacement value would automatically be applied to each match found. If a callable is specified, it would return the new replacement value. In Kenneth's example:

LocusZoom.Layouts.mutate_attr(layout, selector_for_association_fields, (fields_array) => (fields_array.concat(new_items)))

[abought]

A few selector examples that would need to be satisfied for test cases:

Tested in this parser: https://www.npmjs.com/package/jsonpath (it's too big to include in LZ- 200-400kb unminified is significant code bloat, and we don't need eval, just exact match)

> jp.query(layout, '$..color[?(@.scale_function === "if")]')
[
  {
    scale_function: 'if',
    field: 'ld:isrefvar',
    parameters: { field_value: 1, then: '#9632b8' }
  }
]

> jp.query(layout, 'panels[?(@.id === "association")]..scale_function')
[ 'if', 'if', 'if', 'numerical_bin' ]

> jp.query(layout, 'panels[?(@.id === "association")].axes.x.label')
[ 'Chromosome {{chr}} (Mb)' ]

[abought]

Was released in 0.13.3.