Is there, or will there be, a set of test vectors?

[graza]

When implementing standards such as RFCs from the IETF, it can be useful to have a set of test vectors. This would provide someone implementing the standard a target they can aim for.

For example there is the JSON Schema Test Suite on Github. https://github.com/json-schema-org/JSON-Schema-Test-Suite Theirs is language independent and simply provides a set of test cases as JSON files that contain the schema and a set of values to be tested against the schema, along with the expected outcome.

For a JSONPath test suite, a similar approach could be taken with a JSON value with a set of JSON Paths along with expected output.

Kind Regards, Graham

[gregsdennis]

@glyn manages an informal test suite. It's a good starting point, but the spec is moving quickly enough that the suite is difficult to keep updated.

[glyn]

Hi Graham

Thanks for raising this. I was previously working on a Compliance Test Suite ([1]) and a Reference Implementation ([2], in Rust) for JSONPath before circumstances changed and I had to focus my efforts on editing the spec. I still believe these would be valuable resources for implementers and would also help to validate the spec. If anyone reading this message would like to help with either or both of these, please let me know.

The RI has a somewhat fragile dependency on the pest parser which isn't being actively developed. I'd be happy for someone else to start a RI in a different language. One advantage of the pest parser is that the grammar doesn't have any source code in it, which makes it easier to compare to the spec's BNF.

Regards, Glyn

[1] https://github.com/jsonpath-standard/jsonpath-compliance-test-suite [2] https://github.com/jsonpath-standard/jsonpath-reference-implementation

On Fri, 24 Jun 2022 at 04:02, Greg Dennis @.***> wrote:

@glyn https://github.com/glyn manages an informal test suite. It's a good starting point, but the spec is moving quickly enough that the suite is difficult to keep updated.

— Reply to this email directly, view it on GitHub https://github.com/ietf-wg-jsonpath/draft-ietf-jsonpath-base/issues/204#issuecomment-1165144207, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAXF2KY5YZ42RW25K2WD53VQUQKVANCNFSM5ZV6FVWA . You are receiving this because you were mentioned.Message ID: @.*** com>

[graza]

Thanks Greg & Glyn. I've been looking around here at some of the issues / discussions that have been going on. Given all the various implementations there are, is there a sense of how unifying this initiative will be?

I happened on JSONPath initially through the thought that "surely this must be a thing". More recently I discovered that TMForum has adopted it as part of their API design guidelines. But their examples are weak if not wrong (to my eye). So this standard is sorely needed.

My own implementation is, unfortunately, written in a proprietary scripting language. It's a long story but the language is built into a product we market to the telecommunications industry. So TMForum is important to us - compliance with TMForum standards is a way in which we get measured during sales, but also provides a common architecture we can interoperate with.

Anyway, thanks for the links to the jsonpath-standard GitHub project and repositories. I will have a go at running the test suite.

All the best for the weekend, Graham

[cburgmer]

Hey Graham, if you want to get an understanding of how this evolving standard will compare to existing implementations, then https://cburgmer.github.io/json-path-comparison/ might help here. On the right side there with all the other implementations you'll see the compliance test suite Glyn links to above.

In this project there is already a big yaml file with test cases and expected results, although the latter is inferred not from the evolving standard, but currently from a majority vote. This could get you started, if you are looking for double checking your own implementation.

[glyn]

I don't think there's an issue here. Further discussion should take place on the JSONPath mailing list.