Parameterised/templated JSONPath queries

[graza]

At work, I'm proposing JSONPath as part of a rules engine within a TMForum based product catalog system. Although JSONPath is a query language, not intended for use as a rules engine, my rationale for doing this is that JSONPath is already part of the TMForum API design guidelines, so the rules can be shared by the catalog with other components that can execute them with a reasonable expectation that they will be understood by the components consuming the catalog.

Fundamentally, a rule based on JSONPath would have a true/false value, based on whether the JSONPath returns any values when applied to a defined context. An example is a rule to check whether a customer is of type 'Employee'. The context document:

{
  "name": "API_Cust_004",
  "id": "108003",
  "href": "http://localhost/customer/108003", 
  "characteristic": [
    { "name": "CustomerType", "value": "Employee", "valueType": "string" }, 
    { "name": "OpenCasesCount", "value": "1", "valueType": "string" } 
  ]
}

JSONPath:

$.characteristic[?@.name == "CustomerType" && @.value == "Employee"]

As part of this, there is a desire to make the rules more easily used by business users. JSONPath is not the most intuitive for a business user to write. However, if there was a way for a JSONPath query to define input parameters, it could make the rules based on JSONPath easier to use for other customer types.

Different approaches could be adopted, an obvious one being to have a function that brings in the parameter values:

$.characteristic[?@.name == "CustomerType" && @.value == param("customerTypeValue")]

The issue I have with this is that the value of the parameter has to be furnished as part of the catalog of rules so that a consumer of the catalog can evaluate the JSONPath. The param() function is not currently part of the standars.

Another approach I was thinking about, and the impetus for raising this issue, is to have some syntactic provisions made in the standard for allowing some preprocessing to compose a JSONPath query. When the catalog has a rule with a preprocessed element to it, it can perform the substitution before the JSONPath is shared with any other components. A bit like some of the HTML templating languages we see used in web frameworks:

$.characteristic[?@.name == "CustomerType" && @.value == {customerTypeValue}]

In this case the value in braces would be replaced with a value by a preprocessor before it is able to be parsed / evaluated.

Is there something that the standard could/should do to ensure this sort of preprocessing can be successful?

[cabo]

On 1. Aug 2023, at 16:13, Graham Agnew @.***> wrote:

The param() function is not currently part of the standars.

I think the param() function you seem to be suggesting goes beyond what we planned to include in the base JSONPath specification. But the idea of the function extension registry is that it will be very easy to define new function extensions. So what we could to is write up such an extension (i.e., provide the specification that is requested in Section 3.2 1 and have that ready by the time JSONPath has completed IESG review.

Of course there are many other ways to address your specific problem statement. To me, param() sounds like a minimum viable, but quite powerful, solution.

Grüße, Carsten

[glyn]

Another approach would be to create a preprocessor that emits (soon to be) standard JSONPath. Then the preprocessor could implement arbitrary features without the need for standardisation. There may later be a need to standardise the preprocessor input syntax and semantics if there was broad adoption of particular features, but that bridge could be crossed later once it is clear precisely which features those are.

[graza]

Many thanks for the comments Carsten and Glyn.

For a business user and in the long term, a JSONPath query builder user interface, supported by JSONSchema, that emits the fully formed JSONPath might be the way to go. Does such a thing exist already?

To be honest JSONPath might not be the best fit, but it's JSON focussed, soon to be standardised, and used in other aspects of TMForum. But as I write different JSONPath queries, I often feel the need for index() or similar as discussed in #156.

In the short term I will close this as "not planned" but keep my own options open, based on my original post.

[graza]

To be honest JSONPath might not be the best fit, but it's JSON focussed, soon to be standardised, and used in other aspects of TMForum. But as I write different JSONPath queries, I often feel the need for index() or similar as discussed in #156.

Another thought occurred to me: A useful mode of operation for a JSONPath based rules engine would be a filter-selector only query against a document. Such a thing would make it easier to test values of the document root, and benefit from the ability to short-circuit Logical AND and Logical OR operators.

Considering the example from above:

{
  "name": "API_Cust_004",
  "id": "108003",
  "href": "http://localhost/customer/108003", 
  "characteristic": [
    { "name": "CustomerType", "value": "Employee", "valueType": "string" }, 
    { "name": "OpenCasesCount", "value": "1", "valueType": "string" } 
  ]
}

My query to look for $.characteristic values was fine, but logical expressions against a top level object property like $.name is harder because it will return everything, So $[?$.name =~ 'API'] results in the following:

[
  "API_Cust_004",
  "108003",
  "http://localhost/customer/108003", 
  [
    { "name": "CustomerType", "value": "Employee", "valueType": "string" }, 
    { "name": "OpenCasesCount", "value": "1", "valueType": "string" } 
  ]
]

But I really just want a single logical true/false value based on the logical-expr. From a syntax point of view, a logical-only query might resemble just the filter-expr production from the spec. So: ?$.name =~ 'API'. It's a bit like wrapping the input JSON document into a single element array, except that in such a query, the value of $ and @ would be the same value (i.e. $[0] from that single element array).

[cabo]

It is a known limitation of the JSONPath base specification as defined here that selectors can only be applied to child nodes (or descendant nodes) of an input value (see 1). Maybe I can dig out issues later where we discussed this, there were a few.

More generally speaking, your example is not trying to select a part of the input value (here: the target of the filter), but to ascertain its presence; JSONPath as defined here cannot do any transformation of a segment result.

XPath can do all of these, and way more. JSONPath tried to be a simple mechanism. But there is this human tendency to expand the scope over time... I would be very interested if there is another reasonably balanced notch on the scale from JSONPath base today towards the kitchen sink, but that will be another project.