schibsted / jslt

JSON query and transformation language
Apache License 2.0
638 stars 120 forks source link

Variables, Scope and Import #274

Open ronMilne opened 2 years ago

ronMilne commented 2 years ago

I'm passing in a variable prior to calling the template. The primary file (root.jslt) can access the variable.

let identifier = $headers.Identifier

{ "id": $identifier, ... }

Now I have a second jslt file which I import within root

import "client.jslt" as client

When declaring the following within client.jslt

let identifier = $headers.Identifier

I get the exception "No such variable $headers"

if I remove the let statement from the root.jslt (i.e., don't use it in that file) then client.jslt works! It appears that once you use the variable in any file it is no longer available? Creating a common.jslt and including that in both also doesn't work.

Ron

larsga commented 2 years ago

That sounds strange. Can you give me a minimal, complete example that demonstrates the problem?

ronMilne commented 2 years ago

Once I've got these deadlines out the way will do.

ronMilne commented 1 year ago

Apologies for the somewhat late continuation but yep this is a "feature" that we'd like to not have :) Firstly, you need x2 jslt files in file one (clean-client.jslt)

import "jslt/clean-address.jslt" as address

{
"payload":
    address:BuildAddress(.)
    + { "identifier2": "$maxLength" }
}

and file 2 (clean-address.jslt)

def BuildAddress(entity)
    {
        "identifier1": $maxLength
    }

Notice that in file one maxLength is currently in quotes. (the input in the example is irrelevant :)

To execute the jslt I have the following (basically a direct copy'n'paste from the doco)

    public JsonNode RunJsltTransform(String json) throws JsonProcessingException {
        ObjectMapper mapper = new ObjectMapper();
        JsonNode actualObj = mapper.readTree(json);

        Expression exp = Parser.compileResource("jslt/clean-client.jslt");
        JsonNode output = exp.apply(
            // first param is a collection of variables
            Collections.singletonMap("maxLength", (JsonNode) new IntNode(10)),
            actualObj
        );

        return output;
    }

This will execute successfully. Now in file one remove the quotes around $maxLength - what will happen now is you'll get the following exception.

com.schibsted.spt.data.jslt.JsltException: No such variable 'maxLength' at jslt/clean-address.jslt:4:24

If you need anything else please let me know, appreciate you taking a look at this. It's moved up our priority list as we had to merge many many files.

Regards, Ron.

larsga commented 1 year ago

So basically you want to be able to refer to the same constants from the two files?

Why not define shared constants in a third file, as functions with no parameters?

ronMilne commented 1 year ago

importing the same jslt - constants.jslt into both files gives the same error. In constants.jslt I have defined the following

def MaxLength() $maxLength

then simply import it as

import "jslt/constants.jslt" as constants
import "jslt/clean-address.jslt" as address

{
"payload":
    address:BuildAddress(.)
    + { "identifier2": constants:MaxLength() }
}
import "jslt/constants.jslt" as constants

def BuildAddress(entity)
{
    "identifier1": constants:MaxLength()
}
larsga commented 1 year ago

It's not strange that this gives an error, because you don't seem to have defined $maxLength anywhere.

Can't you just do this?

def MaxLength() 25
ronMilne commented 1 year ago

Expression exp = Parser.compileResource("jslt/clean-client.jslt"); JsonNode output = exp.apply( // first param is a collection of variables Collections.singletonMap("maxLength", (JsonNode) new IntNode(10)), actualObj );

still the same error. Even creating a custom function and trying to pass it as a variable fails ... In the example to show the problem we wouldn't be passing 10 as we pass proper variables (such as clientBusinessIdentifier) defined in a request payload. We need this 'identifier' in many parts of the conversion hence we break up the jslt into logical units, e.g., addresses, names, externalIdentfifies etc. each of these needs to also include the clientBusinessIdentifier. The problem is we now have a 400+ line jslt

larsga commented 1 year ago

Right. So let me check that I understand: the problem is that while parameters passed to apply are available as variables in the top-level JSLT expression, they're not available inside imported modules? If so, I agree we should probably change that.

catull commented 1 year ago

@larsga In which way ?

Implicitly by scope ? Thus top scope's variables are visible in lower / nested scopes ? Or explicitly, you have to include variables into some context, which is "global" ? Or third way, perhaps ?

Just curious.

Keep the good work up.

larsga commented 1 year ago

By scope, yes. Basically, one could consider the parameter scope to be the outermost one, so if variables are not found anywhere else they can be looked for there before giving up. Effectively that would be "global", as you say.

(Thanks!)