Python implementation?

[qlyoung]

Hi! I noticed there's a mention of native implementations other than in JS, and was wondering if anyone is working on a Python implementation. I don't see anything on GH, but asking anyway in case I missed it. Embedding V8 in Python to use JSONata sucks :laughing: Cheers

[andrew-coleman]

Not that I know of. I'll leave this open for a while in case anyone knows of one in the works.

There's an extensive compliance test suite if you decide to write a new implementation ;-)

[loganvolkers]

If we manage to get a WebAssembly implementation of JSONata complete, then you could use wasmtime for python interop and we wouldn't need a full separate implementation.

Here's a wasmtime demo for python for markdown parsing: https://github.com/bytecodealliance/wasmtime-demos/tree/master/python

[svet-b]

There is a library, pyjsonata, which is based on Python bindings for JSONata running as C via Duktape. Although circuitous, this approach results in a more compact footprint than alternatives I've come across. It's in pre-release (and hasn't been updated in a fair while) but we've been using it for a while and it appears to work well.

[qlyoung]

@svet-b yes, I wrote that library :wink: https://github.com/qlyoung/pyjsonata

As a note, there's some issues with it, some weird crash when using one of the more uncommon jsonata features if I remember correctly, that only reproduces on non-browser (but still standard) runtimes. I asked about on the jsonata slack at one point and never received a reply. I don't think there's much interest from the maintainers of jsonata in standard runtimes other than Node or a browser unfortunately. It wasn't a Duktape bug as I could reproduce it on multiple runtimes. That's mostly why I never moved it to a 1.0 release.

But I'm glad to hear it's getting some use :)

[svet-b]

@qlyoung apologies for not giving credit where it's due 🙂. I was conscious that perhaps it was something you worked on as a hobby project at some point, and wasn't sure if you're still interested in maintaining it / getting it to a 1.0 release. Maybe it's worth seeing if the JSONata 1.8.4 upstream codebase offers any improvements with respect to the issues you mentioned?

As I said, we've actually been using it quite extensively (for data processing on some Raspberry Pi-based edge devices), and are yet to encounter significant issues. So thank you for the contribution!

The main niggle I have at present is that - as you know - automatic building from source doesn't work. Which means that running it on a Mac for example (as I do for dev work), requires a manual build + local installation. So if you do ever get around to Pythonifying that Makefile, your efforts will not go unnoticed :). (obviously I'd also love to find the time to dig into and do a PR, though it hasn't happened yet)

[qlyoung]

apologies for not giving credit where it's due

Oh not at all, I read your original comment as an update to the issue to let me know that that library was around and thought it was funny that I wrote it, but I guess I misread you :)

... wasn't sure if you're still interested in maintaining it / getting it to a 1.0 release. Maybe it's worth seeing if the JSONata 1.8.4 upstream codebase offers any improvements with respect to the issues you mentioned?

I had plans to use it for a production use case at one point but those never materialized. I can take a look at some point but it seems unlikely. The problem I was seeing in particular was with partial functions, for example:

jsonata('( $first5Capitalized := $substring(?, 0, 5) ~> $uppercase(?);  $first5Capitalized("dogecoin") )').evaluate({})

On spidermonkey and V8 the output was DOGEC but on QuickJS and Duktape it was DOGECOIN. I'm really terrible at JS so I don't have any guesses what could be going wrong here. More complicated partials such as match(?, <regexp>) crashed outright under those runtimes.

The main niggle I have at present is that - as you know - automatic building from source doesn't work.

This is actually a setuptools / python problem as I recall. Python pretends to have good support for C extensions, but in reality there is only decent support for CPython extensions, and your C project needs to be very tightly integrated with setuptools for source builds to work even then.

To go off the rails even more here - I'm glad it's working for you on a raspi because it crashed pip on my model of pi's, since they were soft float pi's and as it turns out, PyPI doesn't have the ability to support multiple ABIs for a single arch, so only armhf platforms are supported for the armv7l arch! And of course the standard Python approach is to vomit stack on the user when a problem arises :roll_eyes:

I wish I could say otherwise but I am unlikely to spend much time on this in the future as the general Python packaging experience was so painful it made me reconsider using Python for serious projects ever again. If you encounter issues in it do open an issue or PR over there though, I'd like to keep it at its current level of usefulness.

[rayokota]

I created a pure Python implementation of JSONata here: https://github.com/rayokota/jsonata-python