"object wrapper" value types (or monkey patching getSymbol?)

[darrencruse]

We are using jsonata with our json api payloads and it's been working well except we do something unusual - we convert the values of our json payload keys to object wrappers i.e. String, Number, Boolean.

The reason is that we attach some meta data as (non-enumerable) properties of these (object wrapped) values in order to access the meta data after jsonata has transformed the json.

So (for one example of our "metadata") our users can assign user friendly names to the json properties that they want to appear as headings in a table, but they define these against the original payloads in the form the apis return them.

(along with other metadata) we attach these "friendly names" to the (object-wrapped) values of the json payloads, then run our jsonata expression, followed by walking what jsonata returns to get back the metadata from the now transformed payload.

This works well in general except the function signature validation that jsonata does assumes primitive values only.

e.g. I was using $toMillis in my jsonata expression but it was throwing an exception since it's first argument was a String object not a string primitive because of our "object wrappers".

stepping into jsonata it looked like I could pretty easily avoid the issue with a small mod to the function "getSymbol" to recognize String/Number/Boolean as equivalent to the corresponding primitives.

Unfortunately I can't see any way to monkey patch my change into the jsonata code, but otoh I'm not sure you guys would want my change in jsonata proper (I recognize what we're doing is unusual we still can't decide if it's genius or insanity :).

I'll paste my little mod below so u can see - would u guys be open to a PR with such a change?

If not would u be open to a change that would introduce some way for me to monkey patch the getSymbol function so I could introduce my change without having to copy jsonata (which I don't want to do if at all possible).

Here was my little test change it's identical to the real getSymbol function except for the three ifs you see checking for String/Number/Boolean:

        var getSymbol = function (value) {
            var symbol;
            if (utils.isFunction(value)) {
                symbol = 'f';
            } else {
                var type = typeof value;
                if (type === 'object' && value instanceof String) {
                  type = 'string';
                }
                if (type === 'object' && value instanceof Number) {
                  type = 'number';
                }
                if (type === 'object' && value instanceof Boolean) {
                  type = 'boolean';
                }
                switch (type) {
                    case 'string':
                        symbol = 's';
                        break;
                    case 'number':
                        symbol = 'n';
                        break;
                    case 'boolean':
                        symbol = 'b';
                        break;
                    case 'object':
                        if (value === null) {
                            symbol = 'l';
                        } else if (Array.isArray(value)) {
                            symbol = 'a';
                        } else {
                            symbol = 'o';
                        }
                        break;
                    case 'undefined':
                    default:
                        // any value can be undefined, but should be allowed to match
                        symbol = 'm'; // m for missing
                }
            }
            return symbol;
        };

sorry for being wordy (wanted to give the context for why this unusual request)

[darrencruse]

have been hoping for a reply - I was worrying maybe I wrote too TLDR much - but I don't see any jsonata team replies in about a month? anybody know or have a guess as to why?

(I was just checking there's not a Discord channel or Google Group or something like that right...)

[cyberwombat]

Seems like there is a slack group - not sure how active though: http://docs.jsonata.org/contributing

[andrew-coleman]

Apologies for slow response, I've been out of action for a while. I'll add this as an enhancement request.

[darrencruse]

@andrew-coleman thanks my one doubt is if marking it an enhancement request means it just queues up and might take a long time to resolve?

(we are hitting this issue in our production code so I was looking for a quick solution)

Assuming it won't be quick we're thinking we'll have to fork jsonata for now but maybe I can figure out a way to minimize our change...

e.g. I was thinking e.g. to move the declaration of getSymbol or assign it to a "JSONata" global or something(?) i.e. where I could reassign it with my customized function...

there's not already something like that is there? (a global like that or some kind of "options" meant for client code to configure settings etc.?)

if we can come up with a tiny change like that for me to "override" what I need maybe I could submit that as a PR even?

[andrew-coleman]

As a quick fix could you JSON.stringify() then JSON.parse() before evaluating the jsonata?

[darrencruse]

that avoids jsonata erroring out since all the values convert back to primitives instead of the Number/String/Boolean object types but unfortunately it defeats the purpose which is to be able to look at the "shape" of the json jsonata returned and regardless of what type of transformation jsonata did still grab the metadata we stuck on the properties of those objects

don't know if it helps but I often think of "flatten" for illustration - you know the kind of flatten that combines the keys down the json tree with underscores to come up with new keys but lifts up the values of what were originally the leaves of the json tree as the values of the newly constructed keys

to me that's a good example to illustrate we have e.g. these "pretty names" (and other stuff) stuck on the values of those leaves of the json, and whether it were a flatten or some other transformation to the json, with the exception of where jsonata creates new values (arithmetic or whatever), this scheme lets us carry this information thru just by walking the json jsonata returns, without specific fore-knowledge of the particular jsonata expression - flatten was just an example.

(again - with the exception of newly generated values e.g. computed values - but that's not typical for us and we do some defaulting in those cases)

don't know if that helps but fwiw

(I wish there was a simple alternative but we've struggled to come up with any good ones! we were originally trying to keep like a parallel "meta data" structure on the side and hand code "transformations" of that structure to kind of match the corresponding transformation of our json payloads - that was really hard and in fact that was before we embraced jsonata we were far more limited in the types of our "transformations" when we were trying to do that)

[andrew-coleman]

Hi Darren, a while back I started working on a 'visual debugger' for the exerciser which will allow you to visually trace data items from the input JSON through to the output structure and conversely visualise how the output was generated from the input. In order to make this work, I wrote a JSON parser that generates the object forms of the primitive types and adds metadata - similar to what you are describing. I was hoping to have had this finished by now, but my day job inevitably gets in the way ;-). I think some of the code could be helpful to you, so I've pushed up the branches (debugger which is based on ast). It's still very much work in progress. The parser itself is in the debugger branch of the exerciser repo (https://github.com/jsonata-js/jsonata-exerciser/blob/debugger/src/json-parse.js) - again this is not production ready yet, but feel free to use it if it's helpful.

[darrencruse]

thanks Andrew I'll take a look sounds interesting

prior to reading your message we'd decided to go ahead and use a fork of jsonata with something like the little change I showed in my first message above

(I'm thinking maybe of introducing a global like e.g. "JSONata.getSymbol = getSymbol" just so I can minimize my change to jsonata proper and externalize my customized override of the "getSymbol" function).

Guessing we're going that route short term but I'll take a look at the debugger branch maybe there's a solution there longer term