StackStorm / st2

StackStorm (aka "IFTTT for Ops") is event-driven automation for auto-remediation, incident responses, troubleshooting, deployments, and more for DevOps and SREs. Includes rules engine, workflow, 160 integration packs with 6000+ actions (see https://exchange.stackstorm.org) and ChatOps. Installer at https://docs.stackstorm.com/install/index.html
https://stackstorm.com/
Apache License 2.0
6.07k stars 749 forks source link

Actions in Mistral workflow succeed, but workflow marked as failed #3549

Closed Mierdin closed 7 years ago

Mierdin commented 7 years ago

Recording this for posterity, as I ran into this once shortly after I started working on StackStorm, and would have loved to see this myself.

I was tinkering with mistral-basic workflow in the examples pack, and converted some of it to use Jinja instead of YAQL. After running it, however, I noticed that - while my action execution succeeded, the workflow itself returned a failure:

vagrant@st2vagrant:/opt/stackstorm/packs/napalm$ cat /opt/stackstorm/packs/examples/actions/workflows/mistral-basic.yaml
version: '2.0'

examples.mistral-basic:
    description: A basic workflow that runs an arbitrary linux command.
    type: direct
    input:
        - cmd
    output:
        stdout: "{{ _.cmd }}"
    tasks:
        task1:
            action: core.local cmd="{{ _.cmd }}"
            publish:
                stdout: "{{ task('task1').result.stdout }}"
vagrant@st2vagrant:/opt/stackstorm/packs/napalm$ st2 run examples.mistral-basic cmd="echo Helllo"
...
id: 595dd67bc4da5f0521ea90c7
action.ref: examples.mistral-basic
parameters:
  cmd: echo Helllo
status: succeeded
start_timestamp: 2017-07-06T06:19:39.787181Z
end_timestamp: 2017-07-06T06:19:44.716185Z
+--------------------------+------------------------+-------+------------+-------------------------------+
| id                       | status                 | task  | action     | start_timestamp               |
+--------------------------+------------------------+-------+------------+-------------------------------+
| 595dd67cc4da5f0521ea90ca | succeeded (0s elapsed) | task1 | core.local | Thu, 06 Jul 2017 06:19:40 UTC |
+--------------------------+------------------------+-------+------------+-------------------------------+
Mierdin commented 7 years ago

A great place to get more information about Mistral workflows is with the mistral CLI:

vagrant@st2vagrant:/opt/stackstorm/packs/napalm$ mistral execution-list
+--------------------+--------------------+--------------------+-------------+-------------------+---------+--------------------+--------------------+---------------------+
| ID                 | Workflow ID        | Workflow name      | Description | Task Execution ID | State   | State info         | Created at         | Updated at          |
+--------------------+--------------------+--------------------+-------------+-------------------+---------+--------------------+--------------------+---------------------+
| da3e9301-0908-4c12 | df80f778-eaae-     | examples.mistral-  |             | <none>            | SUCCESS | None               | 2017-07-06         | 2017-07-06 06:15:02 |
| -996c-126fde664226 | 469f-              | basic              |             |                   |         |                    | 06:15:01           |                     |
|                    | 8db9-c3af01cfd3d4  |                    |             |                   |         |                    |                    |                     |
| 75511a68-e014-4493 | df80f778-eaae-     | examples.mistral-  |             | <none>            | ERROR   | Failed to handle   | 2017-07-06         | 2017-07-06 06:16:20 |
| -9ac4-e122e43987b7 | 469f-              | basic              |             |                   |         | action c...        | 06:16:20           |                     |
|                    | 8db9-c3af01cfd3d4  |                    |             |                   |         |                    |                    |                     |
+--------------------+--------------------+--------------------+-------------+-------------------+---------+--------------------+--------------------+---------------------+
vagrant@st2vagrant:/opt/stackstorm/packs/napalm$ mistral execution-get 75511a68-e014-4493-9ac4-e122e43987b7
+-------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field             | Value                                                                                                                                                      |
+-------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+
| ID                | 75511a68-e014-4493-9ac4-e122e43987b7                                                                                                                       |
| Workflow ID       | df80f778-eaae-469f-8db9-c3af01cfd3d4                                                                                                                       |
| Workflow name     | examples.mistral-basic                                                                                                                                     |
| Description       |                                                                                                                                                            |
| Task Execution ID | <none>                                                                                                                                                     |
| State             | ERROR                                                                                                                                                      |
| State info        | Failed to handle action completion [error=Can not evaluate Jinja expression [expression= task(task1).result.stdout , error='task1' is undefined, data={}], |
|                   | wf=examples.mistral-basic, task=task1, action=st2.action]:                                                                                                 |
|                   | Traceback (most recent call last):                                                                                                                         |
|                   |   File "/opt/stackstorm/mistral/local/lib/python2.7/site-packages/mistral/engine/task_handler.py", line 105, in _on_action_complete                        |
|                   |     task.on_action_complete(action_ex)                                                                                                                     |
|                   |   File "/opt/stackstorm/mistral/local/lib/python2.7/site-packages/osprofiler/profiler.py", line 153, in wrapper                                            |
|                   |     return f(*args, **kwargs)                                                                                                                              |
|                   |   File "/opt/stackstorm/mistral/local/lib/python2.7/site-packages/mistral/engine/tasks.py", line 264, in on_action_complete                                |
|                   |     self.complete(state, state_info)                                                                                                                       |
|                   |   File "/opt/stackstorm/mistral/local/lib/python2.7/site-packages/osprofiler/profiler.py", line 153, in wrapper                                            |
|                   |     return f(*args, **kwargs)                                                                                                                              |
|                   |   File "/opt/stackstorm/mistral/local/lib/python2.7/site-packages/mistral/engine/tasks.py", line 163, in complete                                          |
|                   |     data_flow.publish_variables(self.task_ex, self.task_spec)                                                                                              |
|                   |   File "/opt/stackstorm/mistral/local/lib/python2.7/site-packages/mistral/workflow/data_flow.py", line 211, in publish_variables                           |
|                   |     task_ex.published = expr.evaluate_recursively(data, expr_ctx)                                                                                          |
|                   |   File "/opt/stackstorm/mistral/local/lib/python2.7/site-packages/mistral/expressions/__init__.py", line 96, in evaluate_recursively                       |
|                   |     data[key] = _evaluate_item(data[key], context)                                                                                                         |
|                   |   File "/opt/stackstorm/mistral/local/lib/python2.7/site-packages/mistral/expressions/__init__.py", line 79, in _evaluate_item                             |
|                   |     return evaluate(item, context)                                                                                                                         |
|                   |   File "/opt/stackstorm/mistral/local/lib/python2.7/site-packages/mistral/expressions/__init__.py", line 71, in evaluate                                   |
|                   |     return evaluator.evaluate(expression, context)                                                                                                         |
|                   |   File "/opt/stackstorm/mistral/local/lib/python2.7/site-packages/mistral/expressions/jinja_expression.py", line 140, in evaluate                          |
|                   |     result = JinjaEvaluator.evaluate(patterns[0][1], data_context)                                                                                         |
|                   |   File "/opt/stackstorm/mistral/local/lib/python2.7/site-packages/mistral/expressions/jinja_expression.py", line 89, in evaluate                           |
|                   |     ", data=%s]" % (expression, str(e), data_context)                                                                                                      |
|                   | JinjaEvaluationException: Can not evaluate Jinja expression [expression= task(task1).result.stdout , error='task1' is undefined, data={}]                  |
|                   |                                                                                                                                                            |
| Created at        | 2017-07-06 06:16:20                                                                                                                                        |
| Updated at        | 2017-07-06 06:16:20                                                                                                                                        |
+-------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+

The reason for this error is because when referencing tasks in a Jinja snippet, they must be quoted. This is explained in the Jinja + Mistral doc:

Unlike YAQL, a string in a Jinja expression must be explicitly encapsulated in quotes

So since I changed this to use Jinja instead of YAQL, a correct version of the line in question would be:

stdout: "{{ task('task1').result.stdout }}"