h2non / jsonpath-ng

Finally, a JSONPath implementation for Python that aims to be standard compliant. That's all. Enjoy!
Apache License 2.0
564 stars 85 forks source link

Added fast path to parser for simple name lookup paths #154

Closed mchaput closed 7 months ago

mchaput commented 7 months ago

This patch adds a "fast path" that bypasses the parser and constructs the path directly when it's just a series of named field lookups (e.g. 'foo.bar.baz').

In my tests this gives a ~2500x (!) speedup for parsing lots of simple paths of this type, which are common in our workflows.

Here's my simple test code:

domain = ("alfa", "bravo", "charlie", "delta", "echo", "foxtrot", "golf",
          "hotel", "india", "juliet", "kilo", "lima", "mike", "november",
          "oscar", "papa", "quebec", "romeo", "sierra", "tango")

def random_path(length: int) -> str:
    parts = [random.choice(domain) for i in range(length)]
    return ".".join(parts)

def test_jsonpath(inputs: list[str]) -> float:
    t = perf_counter()
    for p in inputs:
        p = random_path()
        jp = parse_jsonpath(p)
    t = perf_counter() - t
    print(f"{t:0.4f} seconds")
    return t

if __name__ == "__main__":
    inputs = [random_path(4) for _ in range(100)]
    test_jsonpath(inputs)
mchaput commented 7 months ago

Nevermind, there a couple of problems with the regex (e.g. should end with $, and the key matching can be more liberal).