Results do not match other implementations

[cburgmer]

The following queries provide results that do not match those of other implementations of JSONPath (compare https://cburgmer.github.io/json-path-comparison/):

• [ ] $[::] Input:

  [
  "first",
  "second"
  ]

  Expected output:

  ["first", "second"]

  Actual output: NOT_SUPPORTED

  Exception('Parse error at 1:3 near token : (:)')

• [ ] $[:] Input:

  {
  ":": 42,
  "more": "string"
  }

  Expected output:

  []

  Actual output:

  [
  {
    ":": 42,
    "more": "string"
  }
  ]

• [ ] $[0:3:2] Input:

  [
  "first",
  "second",
  "third",
  "forth",
  "fifth"
  ]

  Expected output:

  ["first", "third"]

  Actual output: NOT_SUPPORTED

  Exception('Parse error at 1:5 near token : (:)')

• [ ] $[0:3:1] Input:

  [
  "first",
  "second",
  "third",
  "forth",
  "fifth"
  ]

  Expected output:

  ["first", "second", "third"]

  Actual output: NOT_SUPPORTED

  Exception('Parse error at 1:5 near token : (:)')

• [ ] $[010:024:010] Input:

  [
  0,
  1,
  2,
  3,
  4,
  5,
  6,
  7,
  8,
  9,
  10,
  11,
  12,
  13,
  14,
  15,
  16,
  17,
  18,
  19,
  20,
  21,
  22,
  23,
  24,
  25
  ]

  Expected output:

  [10, 20]

  Actual output: NOT_SUPPORTED

  Exception('Parse error at 1:9 near token : (:)')

• [ ] $[0:4:2] Input:

  [
  "first",
  "second",
  "third",
  "forth",
  "fifth"
  ]

  Expected output:

  ["first", "third"]

  Actual output: NOT_SUPPORTED

  Exception('Parse error at 1:5 near token : (:)')

• [ ] $[1:3:] Input:

  [
  "first",
  "second",
  "third",
  "forth",
  "fifth"
  ]

  Expected output:

  ["second", "third"]

  Actual output: NOT_SUPPORTED

  Exception('Parse error at 1:5 near token : (:)')

• [ ] $[::2] Input:

  [
  "first",
  "second",
  "third",
  "forth",
  "fifth"
  ]

  Expected output:

  ["first", "third", "fifth"]

  Actual output: NOT_SUPPORTED

  Exception('Parse error at 1:3 near token : (:)')

• [ ] $..[0] Input:

  [
  "first",
  {
    "key": [
      "first nested",
      {
        "more": [
          {
            "nested": [
              "deepest",
              "second"
            ]
          },
          [
            "more",
            "values"
          ]
        ]
      }
    ]
  }
  ]

  Expected output (in any order as no consensus on ordering exists):

  ["deepest", "first nested", "first", "more", {"nested": ["deepest", "second"]}]

  Error:

  KeyError(0)

• [ ] $[-2] Input:

  [
  "one element"
  ]

  Expected output:

  []

  Error:

  IndexError('list index out of range')

• [ ] $.*[1] Input:

  [
  [
    1
  ],
  [
    2,
    3
  ]
  ]

  Expected output:

  [3]

  Actual output:

  []

• [ ] $[0] Input:

  {
  "0": "value"
  }

  Expected output:

  []

  Error:

  KeyError(0)

• [ ] $[0] Input:

  "Hello World"

  Expected output:

  []

  Actual output:

  [
  "H"
  ]

• [ ] $['*'] Input:

  {
  "*": "value",
  "another": "entry"
  }

  Expected output (in any order as no consensus on ordering exists):

  ["value"]

  Actual output:

  [
  "entry",
  "value"
  ]

• [ ] $['*'] Input:

  {
  "another": "entry"
  }

  Expected output:

  []

  Actual output:

  [
  "entry"
  ]

• [ ] $..[*] Input:

  {
  "key": "value",
  "another key": {
    "complex": "string",
    "primitives": [
      0,
      1
    ]
  }
  }

  Expected output (in any order as no consensus on ordering exists):

  ["string", "value", 0, 1, [0, 1], {"complex": "string", "primitives": [0, 1]}]

  Actual output:

  [
  "string",
  "value",
  0,
  1,
  1,
  {
    "another key": {
      "complex": "string",
      "primitives": [
        0,
        1
      ]
    },
    "key": "value"
  },
  {
    "complex": "string",
    "primitives": [
      0,
      1
    ]
  }
  ]

• [ ] $[*] Input:

  {
  "some": "string",
  "int": 42,
  "object": {
    "key": "value"
  },
  "array": [
    0,
    1
  ]
  }

  Expected output (in any order as no consensus on ordering exists):

  ["string", 42, [0, 1], {"key": "value"}]

  Actual output:

  [
  {
    "array": [
      0,
      1
    ],
    "int": 42,
    "object": {
      "key": "value"
    },
    "some": "string"
  }
  ]

• [ ] $[key] Input:

  {
  "key": "value"
  }

  Expected output:

  NOT_SUPPORTED

  Actual output:

  [
  "value"
  ]

• [ ] $.[key] Input:

  {
  "key": "value",
  "other": {
    "key": [
      {
        "key": 42
      }
    ]
  }
  }

  Expected output:

  NOT_SUPPORTED

  Actual output:

  [
  "value"
  ]

• [ ] $..[1].key Input:

  {
  "k": [
    {
      "key": "some value"
    },
    {
      "key": 42
    }
  ],
  "kk": [
    [
      {
        "key": 100
      },
      {
        "key": 200
      },
      {
        "key": 300
      }
    ],
    [
      {
        "key": 400
      },
      {
        "key": 500
      },
      {
        "key": 600
      }
    ]
  ],
  "key": [
    0,
    1
  ]
  }

  Expected output (in any order as no consensus on ordering exists):

  [200, 42, 500]

  Error:

  KeyError(1)

• [ ] $[?(@.id==42)].name Input:

  [
  {
    "id": 42,
    "name": "forty-two"
  },
  {
    "id": 1,
    "name": "one"
  }
  ]

  Expected output:

  ["forty-two"]

  Actual output: NOT_SUPPORTED

  JsonPathLexerError('Error on line 1, col 2: Unexpected character: ? ')

• [ ] $[0,2].key Input:

  [
  {
    "key": "ey"
  },
  {
    "key": "bee"
  },
  {
    "key": "see"
  }
  ]

  Expected output:

  ["ey", "see"]

  Actual output: NOT_SUPPORTED

  Exception('Parse error at 1:3 near token , (,)')

• [ ] $.屬性 Input:

  {
  "\u5c6c\u6027": "value"
  }

  Expected output:

  ["value"]

  Actual output: NOT_SUPPORTED

  JsonPathLexerError('Error on line 1, col 2: Unexpected character: 屬 ')

• [ ] $.2 Input:

  {
  "a": "first",
  "2": "second",
  "b": "third"
  }

  Expected output:

  ["second"]

  Actual output: NOT_SUPPORTED

  Exception('Parse error at 1:2 near token 2 (NUMBER)')

• [ ] $.*.bar.* Input:

  [
  {
    "bar": [
      42
    ]
  }
  ]

  Expected output:

  [42]

  Actual output:

  []

• [ ] $.*.* Input:

  [
  [
    1,
    2,
    3
  ],
  [
    4,
    5,
    6
  ]
  ]

  Expected output:

  [1, 2, 3, 4, 5, 6]

  Actual output:

  []

• [ ] $..* Input:

  {
  "key": "value",
  "another key": {
    "complex": "string",
    "primitives": [
      0,
      1
    ]
  }
  }

  Expected output (in any order as no consensus on ordering exists):

  ["string", "value", 0, 1, [0, 1], {"complex": "string", "primitives": [0, 1]}]

  Actual output:

  [
  "string",
  "value",
  [
    0,
    1
  ],
  {
    "complex": "string",
    "primitives": [
      0,
      1
    ]
  }
  ]

• [ ] $..* Input:

  [
  40,
  null,
  42
  ]

  Expected output (in any order as no consensus on ordering exists):

  [40, 42, null]

  Actual output:

  []

• [ ] $.* Input:

  [
  "string",
  42,
  {
    "key": "value"
  },
  [
    0,
    1
  ]
  ]

  Expected output:

  ["string", 42, {"key": "value"}, [0, 1]]

  Actual output:

  []

• [ ] $[?(@.key>42 && @.key<44)] Input:

  [
  {
    "key": 42
  },
  {
    "key": 43
  },
  {
    "key": 44
  }
  ]

  Expected output:

  [{"key": 43}]

  Actual output: NOT_SUPPORTED

  JsonPathLexerError('Error on line 1, col 2: Unexpected character: ? ')

• [ ] $[?(@.key>43 || @.key<43)] Input:

  [
  {
    "key": 42
  },
  {
    "key": 43
  },
  {
    "key": 44
  }
  ]

  Expected output:

  [{"key": 42}, {"key": 44}]

  Actual output: NOT_SUPPORTED

  JsonPathLexerError('Error on line 1, col 2: Unexpected character: ? ')

• [ ] $[?(@['key']==42)] Input:

  [
  {
    "key": 0
  },
  {
    "key": 42
  },
  {
    "key": -1
  },
  {
    "key": 41
  },
  {
    "key": 43
  },
  {
    "key": 42.0001
  },
  {
    "key": 41.9999
  },
  {
    "key": 100
  },
  {
    "some": "value"
  }
  ]

  Expected output:

  [{"key": 42}]

  Actual output: NOT_SUPPORTED

  JsonPathLexerError('Error on line 1, col 2: Unexpected character: ? ')

• [ ] $[?(@['@key']==42)] Input:

  [
  {
    "@key": 0
  },
  {
    "@key": 42
  },
  {
    "key": 42
  },
  {
    "@key": 43
  },
  {
    "some": "value"
  }
  ]

  Expected output:

  [{"@key": 42}]

  Actual output: NOT_SUPPORTED

  JsonPathLexerError('Error on line 1, col 2: Unexpected character: ? ')

• [ ] $[?(@[1]=='b')] Input:

  [
  [
    "a",
    "b"
  ],
  [
    "x",
    "y"
  ]
  ]

  Expected output:

  [["a", "b"]]

  Actual output: NOT_SUPPORTED

  JsonPathLexerError('Error on line 1, col 2: Unexpected character: ? ')

• [ ] $[?(@.key==43)] Input:

  [
  {
    "key": 42
  }
  ]

  Expected output:

  []

  Actual output: NOT_SUPPORTED

  JsonPathLexerError('Error on line 1, col 2: Unexpected character: ? ')

• [ ] $[?(@.key=="Motörhead")] Input:

  [
  {
    "key": "something"
  },
  {
    "key": "Mot\u00f6rhead"
  },
  {
    "key": "mot\u00f6rhead"
  },
  {
    "key": "Motorhead"
  },
  {
    "key": "Motoo\u0308rhead"
  },
  {
    "key": "motoo\u0308rhead"
  }
  ]

  Expected output:

  [{"key": "Mot\u00f6rhead"}]

  Actual output: NOT_SUPPORTED

  JsonPathLexerError('Error on line 1, col 2: Unexpected character: ? ')

• [ ] $[?(@.key=="hi@example.com")] Input:

  [
  {
    "key": "some"
  },
  {
    "key": "value"
  },
  {
    "key": "hi@example.com"
  }
  ]

  Expected output:

  [{"key": "hi@example.com"}]

  Actual output: NOT_SUPPORTED

  JsonPathLexerError('Error on line 1, col 2: Unexpected character: ? ')

• [ ] $[?(@.key=="some.value")] Input:

  [
  {
    "key": "some"
  },
  {
    "key": "value"
  },
  {
    "key": "some.value"
  }
  ]

  Expected output:

  [{"key": "some.value"}]

  Actual output: NOT_SUPPORTED

  JsonPathLexerError('Error on line 1, col 2: Unexpected character: ? ')

• [ ] $[?(@.key=='value')] Input:

  [
  {
    "key": "some"
  },
  {
    "key": "value"
  }
  ]

  Expected output:

  [{"key": "value"}]

  Actual output: NOT_SUPPORTED

  JsonPathLexerError('Error on line 1, col 2: Unexpected character: ? ')

• [ ] $.items[?(@.key1==@.key2)] Input:

  {
  "items": [
    {
      "key1": 10,
      "key2": 10
    },
    {
      "key1": 42,
      "key2": 50
    }
  ]
  }

  Expected output:

  [{"key1": 10, "key2": 10}]

  Actual output: NOT_SUPPORTED

  JsonPathLexerError('Error on line 1, col 8: Unexpected character: ? ')

• [ ] $[?(@.address.city=='Berlin')] Input:

  [
  {
    "address": {
      "city": "Berlin"
    }
  },
  {
    "address": {
      "city": "London"
    }
  }
  ]

  Expected output:

  [{"address": {"city": "Berlin"}}]

  Actual output: NOT_SUPPORTED

  JsonPathLexerError('Error on line 1, col 2: Unexpected character: ? ')

• [ ] $[0,1] Input:

  [
  "first",
  "second",
  "third"
  ]

  Expected output:

  ["first", "second"]

  Actual output: NOT_SUPPORTED

  Exception('Parse error at 1:3 near token , (,)')

• [ ] $[0,0] Input:

  [
  "a"
  ]

  Expected output:

  ["a", "a"]

  Actual output: NOT_SUPPORTED

  Exception('Parse error at 1:3 near token , (,)')

• [ ] $.*['c','d'] Input:

  [
  {
    "c": "cc1",
    "d": "dd1",
    "e": "ee1"
  },
  {
    "c": "cc2",
    "d": "dd2",
    "e": "ee2"
  }
  ]

  Expected output:

  ["cc1", "dd1", "cc2", "dd2"]

  Actual output:

  []

• [ ] $[4,1] Input:

  [
  1,
  2,
  3,
  4,
  5
  ]

  Expected output:

  [5, 2]

  Actual output: NOT_SUPPORTED

  Exception('Parse error at 1:3 near token , (,)')

• [ ] $[ 0 , 1 ] Input:

  [
  "first",
  "second",
  "third"
  ]

  Expected output:

  ["first", "second"]

  Actual output: NOT_SUPPORTED

  Exception('Parse error at 1:5 near token , (,)')

For reference, the output was generated by the program in https://github.com/cburgmer/json-path-comparison/tree/master/implementations/Python_jsonpath-ng.

Also do compare the draft version of the planned standard for JSONPath: https://ietf-wg-jsonpath.github.io/draft-ietf-jsonpath-base/draft-ietf-jsonpath-base.html

[tienhung2812]

Hi @cburgmer , about filter JSON Path, can you check at jsonpath_ng.ext

from jsonpath_ng.ext import parse

jsonpath_expr = parse("$[?(@.key=='value')]")

jsonpath_expr.find([
  {
    "key": "some"
  },
  {
    "key": "value"
  }
])

I found it here https://github.com/h2non/jsonpath-ng/issues/8#issuecomment-349408047

[cburgmer]

Ah, thanks @tienhung2812. The results look indeed... different.

jsonpath_ng.ext seems to improve things significantly. Most previously problematic queries seem to now converge towards a consensus, or the majority answer where no consensus exists, or at least stop failing. Only three queries seem to be "better" in the traditional implementation (have now started to fail or error out):

• https://cburgmer.github.io/json-path-comparison/#filter_expression_with_single_equal
• https://cburgmer.github.io/json-path-comparison/#filter_expression_without_parens
• https://cburgmer.github.io/json-path-comparison/#dot_notation_with_key_named_true

Mind you, currently those links show the traditional implementation. Two of those queries might indicate an invalid syntax.

If the community around jsonpath_ng prefers the jsonpath_ng.ext implementation, we could switch the comparison project over to that.

It would be generally possible to compare both implementations side by side in the comparison project, but that would at the same time negatively impact the consensus building, and allow this library two votes in the process, so I would not be in favour of that solution.