bug: VTL parsing is incompatible with real AppSync VTL templates

[sc0ttdav3y]

Is there an existing issue for this?

• [X] I have searched the existing issues

Current Behavior

I'm running a hello world app to evaluate localstack, and I have gotten to a point where my VTL resolver templates are being evaluated.

Unfortunately, I'm getting errors that appear to be Python errors when running my VTL.

I've encountered two so far, but it suggests to me that VTL is emulated in Python and not compatible with AWS' implementation.

I can't find any existing documentation or other issues to confirm this, which is why I'm writing this. I'm trying to get a sense of the differences in this VTL implementation, so I can work out how to proceed with my VTL templates.

Some examples are below.

Expected Behavior

I'd expect a valid VTL template that runs on AWS to run on localstack without modification.

As a realist, I am okay if there are some differences, but I'd hope there would be a list somewhere to avoid whack-a-mole development.

How are you starting LocalStack?

With the localstack script

Steps To Reproduce

How are you starting localstack (e.g., bin/localstack command, arguments, or docker-compose.yml)

GRAPHQL_ENDPOINT_STRATEGY=domain DEBUG=1 localstack start

Client commands (e.g., AWS SDK code snippet, or sequence of "awslocal" commands)

More details of my environment can be found https://github.com/localstack/localstack/issues/8656

Environment

- OS: MacOS Ventura 13.4.1 on an M2
- LocalStack: 2.1.1 with pro license

Anything else?

Examples of errors:

Error in template '<string>' at position 217-297 in expression: #if($resultObj.status >= 400)\n  $util.error($resultObj.message, \"backend\")\n#end\n\n
TypeError: '>=' not supported between instances of 'str' and 'int'

When evaluating valid VTL:

...
#if($resultObj.status >= 400)
    $util.error($resultObj.message, \"backend\")
#end
...

And this:

Error in template '<string>' at position 203-229 in expression: ($resultObj.site = $site)\n\n
TypeError: 'str' object does not support item assignment

when evaluating a valid VTL template containing this:

...
## Parse the JSON response and add 'site' to the top level for sub-resolvers
#set($resultObj = $util.parseJson($ctx.result.body))
#set($resultObj.site = $site)
...

Analysis and guesswork

I'm not a Python developer, but these appear to by Python errors. Correct me if I'm wrong, it's just a guess. My guess is that python's type system is interfering with the aim of emulating a loosely types language in VTL.

In order to resolve these issues, I'd need to know how to write VTL code to overcome Python incompatibilities, which is doubly hard for me given a) I'm not a Python dev, and b) the VTL docs over at AWS are unkind and the language a little esoteric. It would be whack-a-mole on steroids.

Thanks in advance for your help, and I respect your hard work in getting this far with localstack — I recognise the idea of localstack is immense. Given the AppSync component of localstack is payware via the pro licence, it would be awesome it could be more like real VTL, so I could run real VTL templates in it.

Thanks again, Scott

[simonrw]

Hi @sc0ttdav3y, thanks for reporting this issue.

Your assessment is correct - we use airspeed to execute our VTL templates. Can you expand on your templates? What is resultObj in this case, and how are you setting the status variable?

Presumably this template is attached to a resolver that wraps an HTTP data source. Is that correct? In which case, is there are reason you are not checking the status code with $ctx.result.statusCode as per the AWS docs?

Do you have a complete (though minimal) reproducable example that we can use to recreate this issue on our side?

[skymasseronir]

Hi guys, I am having a problem similar to what you describe.

In my case it would seem that the engine interpreting the VTL cannot parse the new lines (\n).

In detail the following VTL produces error:

#set( $modelObjectKey = {
  "userId": $util.dynamodb.toDynamoDB($ctx.args.input.userId),
  "domain": $util.dynamodb.toDynamoDB($ctx.args.input.domain)
} )

{
  "version": "2018-05-29",
  "operation": "PutItem",
  "key": #if( $modelObjectKey ) $util.toJson($modelObjectKey) #else {
  "id":   $util.dynamodb.toDynamoDBJson($ctx.args.input.id)
} #end,
  "attributeValues": $util.dynamodb.toMapValuesJson($context.args.input),
  "condition": $util.toJson($condition)
}

{
    "data": {
        "createUser": null
    },
    "errors": [
        {
            "message": "line 1, column 25: expected expression in ext assignment, got: {\n  \"userId\": $util.dynamodb.toDynam ... ...",
            "locations": [
                {
                    "line": 2,
                    "column": 5
                }
            ],
            "path": [
                "createUser"
            ],
            "errorType": "TemplateSyntaxError"
        }
    ]
}

Whereas if I remove the new lines from the set directive, the problem does not occur:

#set( $modelObjectKey = {  "userId": $util.dynamodb.toDynamoDB($ctx.args.input.userId),  "domain": $util.dynamodb.toDynamoDB($ctx.args.input.domain)} )

{
  "version": "2018-05-29",
  "operation": "PutItem",
  "key": #if( $modelObjectKey ) $util.toJson($modelObjectKey) #else {
  "id":   $util.dynamodb.toDynamoDBJson($ctx.args.input.id)
} #end,
  "attributeValues": $util.dynamodb.toMapValuesJson($context.args.input),
  "condition": $util.toJson($condition)
}

Below few details about the environment (running on Pro tier):

LocalStack version: 2.2.1.dev20230814093317
LocalStack build date: 2023-08-14
LocalStack build git hash: e342fa9

MacOS 13.4.1 (c) on i7

Unfortunately, this is an annoying problem because most resolvers are autogenerated by Amplify, which evidently goes to do some formatting of the code before generating VTLs.

[sc0ttdav3y]

@skymasseronir I couldn't get localstack's AppSync simulator to work for me, and I ultimately found success by using the serverless-appsync-simulator serverless plugin, which under the hood uses the amplify-appsync-simulator npm package. This still had some gotchas, but I managed to get through them.

No disrespect to the amazing efforts and ambitions of localstack of course; it is trying to solve a much larger problem and I'm interested in returning to it when time permits. But I'm very time-poor right now, and in the end the Ampify simulator got me up and running.

[skymasseronir]

Hi @sc0ttdav3y, we are just considering doing a full migration to LocalStack Pro to drop the serverless-appsync-simulator plugin.

I hope someone will consider these issues on AppSync service :)

Thanks for your feedback!

[whummer]

Hi @skymasseronir @sc0ttdav3y , quick update - we have now added a couple of enhancements to our VTL implementation and increased parity with VTL in AWS.

In particular, the example template from @skymasseronir above should now render properly. When you get a chance, can you please pull the latest Docker image and give it another try? Please keep us posted on how it goes. Thanks for your help!

[skymasseronir]

Hi @whummer, I confirm that the issue seems to have been resolved! Thank you! 🚀

[sc0ttdav3y]

Sounds promising, thanks @whummer. I'm not sure if my issues were the same as @skymasseronir, but fingers crossed.

Unfortunately I don't have time right now to test this out, but I wanted to say thanks, and I'll take another look as soon as I can and report back.