Efficient and simple API

[planetis-m]

From the proposal https://github.com/planetis-m/jsonecs/issues/8

The API on top should encourage efficient operations (don't use random access, iterate over the tree instead).

Luckily a proposed standard is already out for us to draw inspiration from https://datatracker.ietf.org/doc/html/rfc6901 and https://datatracker.ietf.org/doc/html/rfc6902

its value MUST be one of "add", "remove", "replace", "move", "copy", or "test"

So be it. API and example usage:

type
  JsonPtr* = distinct string

proc add*(x: var JsonTree; path: JsonPtr; value: sink JsonTree)
proc remove*(x: var JsonTree; path: JsonPtr)
proc replace*(x: var JsonTree; path: JsonPtr, value: sink JsonTree)
proc copy*(x: var JsonTree; `from`, path: JsonPtr)
proc move*(x: var JsonTree; `from`, path: JsonPtr)
proc test*(x: JsonTree; path: JsonPtr, value: JsonTree): bool

var x = %*{
  "a": [1, 2, 3],
  "b": 4,
  "c": [5, 6],
  "d": {"e": [7, 8], "f": 9}
}

add x, JsonPtr"/a/-", %*[5,6]
# """{"a":[1,2,3,[5,6]],"b":4,"c":[5,6],"d":{"e":[7,8],"f":9}}"""

remove x, JsonPtr"/d/e/1"
# """{"a":[1,2,3,[5,6]],"b":4,"c":[5,6],"d":{"e":[7],"f":9}}"""

replace x, JsonPtr"/b", %*"foo"
# """{"a":[1,2,3,[5,6]],"b":"foo","c":[5,6],"d":{"e":[7],"f":9}}"""

copy x, JsonPtr"/b", JsonPtr"/d/f"
# """{"a":[1,2,3,[5,6]],"b":"foo","c":[5,6],"d":{"e":[7],"f":"foo"}}"""

move x, JsonPtr"/c", JsonPtr"/b"
# """{"a":[1,2,3,[5,6]],"b":[5,6],"d":{"e":[7],"f":"foo"}}"""

assert test(x, JsonPtr"/d", %*{"e": [7], "f": "foo"})

Shorter explanation and available libraries at http://jsonpatch.com/

[planetis-m]

# direct match
traverse x, "contact" / "phone":
  discard
# also matches kind
traverse x, (JString, "contact" / "phone"): 
  discard
# is leaf?
traverse x, ({JNull..JString}, "contact" / "phone"):
  discard
# start from the root?
traverse x, / "contact" / "phone":
  discard
traverse x, (Jstring, "emp_data" * "employee" / "name": # * matches all
  discard
# a more refined pattern:
traverse x, (JArray, "emp_data") * (JObject, "employee") / (JString, "name"):
  discard
# with array indices
traverse x, (JArray, "contact") / 0: # contact array, first index
  discard

macro traverse(x: JsonTree, pattern: untyped, body: untyped): untyped

[planetis-m]

As a forStmt macro:

for i in traverse(x, "contact"/"phone"):
  i
# also matches kind
for i in traverse(x, (JString, "contact"/"phone")):
  i
# is leaf?
const JLeaf = {JNull..JString}
for i in traverse(x, (JLeaf, "contact"/"phone")):
  i
# start from the root?
for i in traverse(x, /"contact"/"phone"):
  i
# a more refined pattern:
for i in traverse(x, (JArray, "emp_data") * (JObject, "employee") / (JString, "name")):
  i
# with array indices
for i in traverse(x, (JArray, "contact") / [0]): # contact array, first index
  i
# ranges within array notation?
for i in traverse(x, (JArray, "contact") / [0..2]): # first three elements
  i
# top level is an array
for i in traverse(x, [0..1]): # first two elements
  i

with pattern: typed we can get type information for each node in the pattern.

[planetis-m]

Interesting reads: http://jsonpatch.com/ https://www.sqlite.org/json1.html

[planetis-m]

Might still be needed:

proc len*(x: JsonTree; path: JsonPtr): int
proc kind*(x: JsonTree; path: JsonPtr): JsonNodeKind
proc extract*(x: JsonTree; path: JsonPtr): JsonTree

assert len(x, JsonPtr"/b") == 2
assert kind(x, JsonPtr"/d/e") == JArray
assert $extract(x, JsonPtr"/d") == """{"e":[7],"f":"foo"}"""

Instead of this bloat:

proc getStr*(x: JsonTree, path: JsonPtr, default: string = ""): string
proc getInt*(x: JsonTree, path: JsonPtr, default: int = 0): int
proc getBiggestInt*(x: JsonTree, path: JsonPtr, default: BiggestInt = 0): BiggestInt
proc getFloat*(x: JsonTree, path: JsonPtr, default: float = 0.0): float
proc getBool*(x: JsonTree, path: JsonPtr, default: bool = false): bool
# Iterators
iterator items*(x: JsonTree; path: JsonPtr): JsonTree
iterator pairs*(x: JsonTree; path: JsonPtr): (lent string, JsonTree)
# a functional approach:
proc each[T]*(x: JsonTree; path: JsonPtr; op: proc (x: JsonTree): T {.closure.}): seq[T]
proc tree[T]*(x: JsonTree; path: JsonPtr; op: proc (x: JsonTree): T {.closure.}): seq[T]

...which might still need to expose JsonNode in order to make operations faster. Replace them with just the to macro:

proc fromJson*[T](x: JsonTree; path: JsonPtr; t: typedesc[T]): T
proc toJson*[T](x: T): JsonTree

[planetis-m]

Problems: test is not equivalent to hasKey. There still needs to be a way to iterate over fields/elements. Similar to eminim.jsonItems ForStmt macros:

# iterators
iterator items*(x: JsonTree; path: JsonPtr; t: typedesc[T]): T
iterator pairs*(x: JsonTree; path: JsonPtr; t: typedesc[T]): (lent string, T)

# recursive iterators
iterator itemsRec*(x: JsonTree; path: JsonPtr; t: typedesc[T]): T
iterator pairsRec*(x: JsonTree; path: JsonPtr; t: typedesc[T]): (lent string, T)

[planetis-m]

Syntaxes tried:

1. /"a"/1/_ needs to be inside a macro context, _ doesn't compile. It's a bad idea to turn everything into macros, unless it's a simple macro jptr(path: untyped): untyped that returns JsonPtr. Usage: jptr(/"a"/1/_)
2. /"a"/1/"-" with a declaration like:

   proc `/`*(a: JsonPtr, b: string): JsonPtr = JsonPtr(a.string & "/" & escapeJsonPtr(b))
   proc `/`*(a: string): JsonPtr = JsonPtr("/" & escapeJsonPtr(a))
   proc `/`*(a: JsonPtr, b: int): JsonPtr = JsonPtr(a.string & "/" & $b)
   proc `/`*(a: int): JsonPtr = JsonPtr("/" & $a)

   Seems OK. We take advantage of the fact there is no unary / in std/os and avoid naming collisions.

3. A simple template does the trick. template `$*`*(s: string): JsonPtr = JsonPtr(s) Usage: $*"/a/-" Edit Apparently this works:

   const _* = "-"
   let a: JsonPtr = /"a"/"b"/_

   Meh looks like a bad idea.

[planetis-m]

JSON pointer implementations that seem not to use split:

https://github.com/hitodama/libjsonp https://github.com/dolmen-go/jsonptr https://github.com/lestrrat-go/jspointer

Nim implementation: https://github.com/hnicke/jsonpatch.nim

[planetis-m]

JSON Pointer have been implemented. However splitting them in runtime doesn't come cheap, so a macro solution should be considered.

[planetis-m]

Done.