Serializer?

[loganvolkers]

I'm looking to create an editor that can visually build simple JSONata expressions.

The idea is:

• On load, parse the string expressions to an AST
• Build the UI components based on the AST
• On save, serialize the AST to a string, and save it

The first two steps are pretty straightforward via jsonata(text).ast(), but a serializer seems to be something that doesn't exist in the source code.

Is there another serializer library somewhere that I'm missing? Is there interest in adding a serializer (e.g. jsonata(text).toString())?

It seems at present my workaround is to write a custom serializer for the subset of syntax and AST that I'm able to support with the UI.

[Allam76]

That is what I did as well. You can use optimiseAst or evaluate methods to see how jsonata traverse the AST internally.

On a general note, it is hard to avoid the custom serialiser in my experience. Just a small change in wanted behaviour and existing ones tend to no longer be useful.

Cheers Martin

[andrew-coleman]

Just be aware that the AST is designed for internal use, and might change between releases. Indeed the AST structure for a path expression changed quite a bit in v1.7.0.

[loganvolkers]

That is what I did as well. You can use optimiseAst or evaluate methods to see how jsonata traverse the AST internally.

Thanks @Allam76 that's the path that I went down. You can see the work in progress. It doesn't implement the entire AST yet.

[loganvolkers]

Just be aware that the AST is designed for internal use, and might change between releases. Indeed the AST structure for a path expression changed quite a bit in v1.7.0.

Thanks for the warning @andrew-coleman. I was looking at the typescript definitions in the 2.0 branch and noticed a lot of changes there, too.

Would you consider supporting tooling a goal of this project?

We're looking to build out a bunch of tooling around JSONata. Would introduce major backwards incompatible AST changes between 1.7.0 and 1.7.1, or would we only see that between 1.7 and 1.8 for example?

Would you be open to pull requests for some tooling? Or would you rather see these as separate projects?

• adding a serializer
• ability to turn an AST into an evaluatable expression (related)
• more complete AST type definitions

[andrew-coleman]

Would you consider supporting tooling a goal of this project?

Yes. There are a number of people who have expressed an interest in this in some form. This has ranged from simple content assist in an editor to a drag-and-drop graphical data mapper for generating transformations. I see you've joined the slack channel - it might be worth posting a proposal there to see if we can find a consensus on what to build and who would be willing to contribute.

Given enough interest, we could add a new repo in https://github.com/jsonata-js as we did with the exerciser.

Would introduce major backwards incompatible AST changes between 1.7.0 and 1.7.1, or would we only see that between 1.7 and 1.8 for example?

I don't plan to make any AST changes in a maintenance release - it would only happen if it was needed to fix any bugs in the language. I will need to add a new node type to the AST in order to support a 'parent operator' #299, which I'm hoping to do for v1.8, but even this shouldn't cause any breaking changes.

[loganvolkers]

Thanks for the response @andrew-coleman.

First thing on my list is to fix the typescript definitions for the AST, as they are incomplete and therefore cause errors.

I'll start up a conversation in slack.

[loganvolkers]

Work on the serializer has resulted in a spinoff module: jsonata-ui-core -- https://github.com/jsonata-ui/jsonata-ui-core

The serializer is still incomplete. Once it supports 100% of JSONata expressions (measured via the existing jsonata test suite), then I'll open a pull request.