Lookup in deeply nested JSON, an abbreviated syntax for map:find

[benibela]

In XML, you can select all X nodes with an abbreviated syntax //X

There is no abbreviated syntax for JSON

I propose to add a ?? syntax. Like / is doubled for //, it doubles the ? lookup operator.

The syntax is basically the same as for ?:

[200] UnaryLookupRecursion ::= "??" KeySpecifier [143] LookupRecursion ::= "??" KeySpecifier [144] KeySpecifier |::= NCName | IntegerLiteral | StringLiteral | VarRef | ParenthesizedExpr | "*"

For the semantic it can call map:find, except for * and varref:

Unary variant:

??"string"    becomes map:find(. , "string")
??NCName      becomes map:find( ., "NCName")
??123         becomes map:find(., 123)

??*           Recursively every member/value of every array/map underneath .
              E.g. for `[{"a": {"x": 123}}, 456]`: `{"a": {"x": 123}}, {"x": 123}, 123, 456`

??$varref     calls ?$varref on every nested array/map.
              Like (.,??*)?$varref   (except for type errors)

Postfix variant:

E??S would be E!??S if it is atomic, or let $s := data(S) return E!??$s if S is parenthesized

This probably conflicts with #171

[michaelhkay]

This is essentially a duplicate for issue #262, and I propose to close it as such.

There's no point in having a shorthand for map:find(), because the function is effectively useless. The analogy with //x for nodes is a false one, because nodes allow navigation to siblings and ancestors. map:find() gives you a value with no context, it allows you for example to find all the values of "location" anywhere in the tree, but there's almost nothing you can do with the information.

[benibela]

But #262 is about a function, and this is about a syntax

map:find is bad, because it is too verbose. map:search is even more verbose.

From the 262 example, ?container?name?identifier, might just be written as ??identifier to save 14 characters, almost half of the query. map:find would only save 2 characters, and map:search would save none.

And you can use this syntax to do things map:find cannot do. With ??("a", "b"), you would get two values. map:find would need to be called twicemap:find(., "a"), map:find(., "b")but even then it is something else, since map:finds returns what//@a, //@bwould return for XML, while ?? would interleave them like//@a | //@b` would for XML.

Sometimes JSON documents have optional arrays. The document is either {"x": y} or [{"x": y}]. Then you need map:find or a complex query if (. instance of array(*)) then ?*?x else ?x , but it can be simplified to ??x

Sometimes JSON documents do not use keys, but string ids. Like rather than writing {"foo": ..}, they have {"id": "foo", "value": .. } . Then it is hard to find without ??, but with it, you can write ??* [. instance of map(*) and ?id = "foo"]?value

[ChristianGruen]

I more and more like the idea of introducing an operator for descendant lookups. It’s true that map:find can already be used today, but the resulting code is usually not very readable:

($map?root => map:find('items') => map:find('names'))[map:find(., 'id') = '123']

vs.

$map?root??items??names[??id = '123']

It’s also true that there’s currently no way to access siblings and ancestors. On the other hand, most lookups in practice are forward-only, and many others can be rewritten. I still believe that map:search will be helpful if forward traversal is not sufficient.

---

I had similar thoughts some time ago before I proposed ?? as operator for the ternary conditional operator (https://github.com/expath/xpath-ng/pull/7, now #171). As we’re discussing a slightly more compact syntax for if expressions today – #284 – it might be an overdose to offer three alternatives altogether.

[ChristianGruen]

Sometimes JSON documents have optional arrays

@benibela #115 comes with a proposal to tackle that case.

[michaelhkay]

If the changes suggested in issue #334 are made, then the ?? operator makes a lot more sense, enabling you to write things like $json??firstName[.='Michael']¶parent?lastName, or alternatively $json??lastName[¶parent?firstName='Michael']

[michaelhkay]

Reading the proposal again, I see one might allow $json??*[?firstName='Michael']?lastName. Here the ??* operator no longer corresponds to map:find, but to something that does "find all descendant maps and arrays". This can be defined without any dependency on the "transient properties" proposed in issue #334.

[ChristianGruen]

Incorporated in the spec, can be closed.