Parsing bug of yqal and jinja

[chain312]

SUMMARY

When the input input contains the format of the yaql expression <% XXX% > or the jinja expression contains {{XXX}}, the input is considered to be part of the expression, and the content in the input has the format of the expression, but it is not a complete expression, so that the script will run with an error.

STACKSTORM VERSION

st2 3.8.0

OS, environment, install method

Post what OS you are running this on, along with any other relevant information/

Vagrant

Steps to reproduce the problem

I built a new playbook to test it.

version: 1.0
tasks:
  # [39, 67]
  task1:
    action: core.echo
    next:
      # #629e47
      - do:
          - task2
        when: <% ctx(data)[name]="2" %>
        publish:
          - key: success
      # #d1583b
      - do:
          - task3
        when: <% ctx(data)[name]!="2" %>
        publish:
          - key: success
    input:
      message: aaa
  # [139, 217]
  task2:
    action: core.echo
    input:
      message: <% ctx(data)[name]%>
  # [489, 217]
  task3:
    action: core.echo
    input:
      message: <% ctx(data)[name]%>
input:
  - data

The input to the playbook is {"name": "<% @% >"}

Expected Results

The playbook only executes the task1 output aaa, but does not report an error

Actual Results

{
  "output": null,
  "errors": [
    {
      "type": "error",
      "message": "YaqlEvaluationException: Unable to evaluate expression '<% @ %>'. YaqlLexicalException: Lexical error: illegal character '@' at position 0"
    }
  ]
}

Jinja has the same problem, please tell me how to fix it

[nzlosh]

There seems to be some confusion. You keep mentioning playbook but the examples you show are Orquesta workflows. I don't know if you're expecting Orquesta workflows to work like ansible playbooks, but they are not the same thing and won't work in the same way.

YAQL and Jinja expressions can be defined as parameters, they're evaluated and resolved before executing the workflow. Their result is used as the value of the input parameter.

In the example above, the input YAQL expression is invalid, so an error is produced and the workflow is not executed. The fix is to write a valid YAQL expression or if you really want to pass "@" into the workflow just write it as a string {"name": "@"} in the dictionary.

[chain312]

There seems to be some confusion. You keep mentioning playbook but the examples you show are Orquesta workflows. I don't know if you're expecting Orquesta workflows to work like ansible playbooks, but they are not the same thing and won't work in the same way.

YAQL and Jinja expressions can be defined as parameters, they're evaluated and resolved before executing the workflow. Their result is used as the value of the input parameter.

In the example above, the input YAQL expression is invalid, so an error is produced and the workflow is not executed. The fix is to write a valid YAQL expression or if you really want to pass "@" into the workflow just write it as a string {"name": "@"} in the dictionary. @nzlosh I'm sorry I have a problem with my expression, but what I think is unreasonable is that the <% > contained in the input will be considered an expression, which is obviously not safe, although I haven't found out how to attack this application. I know the implementation here orquesta/expressions/yql.py:146, but I don't know how to change the code so that he doesn't parse the input <% %>.

[nzlosh]

I don't understand what you're attempting to achieve and can't help any further.

[guzzijones]

Not a bug

[eidorb]

@chain312 I think I understand what you're saying.

If the result of evaluating a YAQL expression contains a YAQL expression, then Orquesta attempts to recursively evaluate the YAQL expression instead of treating it as data, resulting in a YAQL evaluation error.

Take a look here for some potential workarounds: https://github.com/StackStorm/st2/issues/4636

[chain312]

@eidorb Hello, mate. Thank you for your response. I have now fixed the issue by modifying the code in the Orquesta project. I have implemented regular expressions to filter out YAQL expressions. If a YAQL expression does not match the regular expression pattern, it will not be parsed as a YAQL expression.