search

StackStorm / orquesta

Orquesta is a graph based workflow engine for StackStorm. Questions? https://github.com/StackStorm/st2/discussions
https://docs.stackstorm.com/orquesta/
Apache License 2.0
98 stars 39 forks source link

Exception running workflow - 'ValueError: malformed node or string: <_ast.BinOp object at 0x7f80d29946d8>' #186

Open nzlosh opened 4 years ago

nzlosh commented 4 years ago

I haven't been able to find where this error is being generated from. As far as I can tell the action follows format and syntax correctly.

Action from workflow that is failing

  send_maintenance_start:
    action: email.send_email
    input:
      account: smarthost
      email_from: "no-reply@XXXXXX"
      email_to: "carlos@XXXXXX"
      subject: Incident detected on <% ctx("hostname") %>
      message: Automatic reboot of <% ctx("hostname") %> to recover from host down.
    next:
      - do: disable_users

Exception from workflowegine log.

2020-01-17 23:33:43,532 140191264317064 INFO (unknown file) [-] [5e22445438b7e59d28ade4fb] Mark task "send_maintenance_start", route "0", in conductor as running.
2020-01-17 23:33:43,580 140191264317064 INFO (unknown file) [-] [5e22445438b7e59d28ade4fb] Requesting execution for task "send_maintenance_start", route "0".
2020-01-17 23:33:43,580 140191264317064 INFO (unknown file) [-] [5e22445438b7e59d28ade4fb] Processing task execution request for task "send_maintenance_start", route "0".
2020-01-17 23:33:43,589 140191264317064 INFO (unknown file) [-] [5e22445438b7e59d28ade4fb] Task execution "5e22445738b7e59c9835e3ff" created for task "send_maintenance_start
", route "0".
2020-01-17 23:33:43,599 140191264317064 WARNING (unknown file) [-] Determining if exception <class 'ValueError'> should be retried.
2020-01-17 23:33:43,599 140191264317064 WARNING (unknown file) [-] Determining if exception <class 'ValueError'> should be retried.
2020-01-17 23:33:43,599 140191264317064 ERROR (unknown file) [-] [5e22445438b7e59d28ade4fb] Failed action execution(s) for task "send_maintenance_start", route "0". malforme
d node or string: <_ast.BinOp object at 0x7f80d29946d8>
Traceback (most recent call last):
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/util/casts.py", line 36, in _cast_object
    return json.loads(x)
  File "/usr/lib/python3.6/json/__init__.py", line 354, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.6/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.6/json/decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
nzlosh commented 4 years ago

This error may be specifically related to the email.send_email pack since when I comment it out, the workflow runs correctly. I'll try to get more information.

nzlosh commented 4 years ago

FYI environment is Ubuntu Bionc. st2 3.1.0, on Python 3.6.9

MongoDB shell version v4.0.14
git version: 1622021384533dade8b3c89ed3ecd80e1142c132
OpenSSL version: OpenSSL 1.1.1  11 Sep 2018
allocator: tcmalloc
modules: none
build environment:
    distmod: ubuntu1804
    distarch: x86_64
    target_arch: x86_64
nzlosh commented 4 years ago

Sorry, I just realised that I didn't provide the full exception, the first exception was raised while processing another exception. The full trace is as follows:

2020-01-20 06:55:53,317 140248351184592 ERROR (unknown file) [-] [5e254ef138b7e5701564ce3f] Failed action execution(s) for task "send_maintenance_finished", route "0". malformed node or string: <_ast.BinOp objec
t at 0x7f8e1d2054e0>
Traceback (most recent call last):
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/util/casts.py", line 36, in _cast_object
    return json.loads(x)
  File "/usr/lib/python3.6/json/__init__.py", line 354, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.6/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.6/json/decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/services/workflows.py", line 521, in request_task_execution
    request_action_execution(wf_ex_db, task_ex_db, st2_ctx, ac_ex_req, delay=ac_ex_delay)
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/retrying.py", line 49, in wrapped_f
    return Retrying(*dargs, **dkw).call(f, *args, **kw)
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/retrying.py", line 206, in call
    return attempt.get(self._wrap_exception)
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/retrying.py", line 247, in get
    six.reraise(self.value[0], self.value[1], self.value[2])
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/six.py", line 693, in reraise
    raise value
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/retrying.py", line 200, in call
    attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/retrying.py", line 49, in wrapped_f
    return Retrying(*dargs, **dkw).call(f, *args, **kw)
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/retrying.py", line 206, in call
    return attempt.get(self._wrap_exception)
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/retrying.py", line 247, in get
    six.reraise(self.value[0], self.value[1], self.value[2])
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/six.py", line 693, in reraise
    raise value
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/retrying.py", line 200, in call
    attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/services/workflows.py", line 615, in request_action_execution
    ac_ex_ctx
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/util/param.py", line 312, in render_live_params
    live_params = _cast_params_from(params, context, [action_parameters, runner_parameters])
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/util/param.py", line 245, in _cast_params_from
    result[name] = _cast(context[name], param_schema)
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/util/param.py", line 76, in _cast
    return cast(v)
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/util/casts.py", line 38, in _cast_object
    return ast.literal_eval(x)
  File "/usr/lib/python3.6/ast.py", line 85, in literal_eval
    return _convert(node_or_string)
  File "/usr/lib/python3.6/ast.py", line 77, in _convert
    left = _convert(node.left)
  File "/usr/lib/python3.6/ast.py", line 84, in _convert
    raise ValueError('malformed node or string: ' + repr(node))
ValueError: malformed node or string: <_ast.BinOp object at 0x7f8e1d2054e0>

I tested the same action being called using Action-Chain which produced the following error:

2020-01-20 11:05:04,399 139887214753032 ERROR (unknown file) [-] Failed to run action.
Traceback (most recent call last):
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/util/casts.py", line 36, in _cast_object
  File "/usr/lib/python3.6/json/__init__.py", line 354, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.6/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.6/json/decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/models/utils/action_param_utils.py", line 129, in cast_params
    params[k] = cast(v)
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/util/casts.py", line 38, in _cast_object
    LOG.critical("TOTO {} {}".format(type(x), x))
  File "/usr/lib/python3.6/ast.py", line 85, in literal_eval
    return _convert(node_or_string)
  File "/usr/lib/python3.6/ast.py", line 77, in _convert
    left = _convert(node.left)
  File "/usr/lib/python3.6/ast.py", line 84, in _convert
    raise ValueError('malformed node or string: ' + repr(node))
ValueError: malformed node or string: <_ast.BinOp object at 0x7f3a07b07b00>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2actions/container/base.py", line 127, in _do_run
    (status, result, context) = runner.run(action_params)
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/action_chain_runner/action_chain_runner.py", line 304, in run
    return self._run_chain(action_parameters)
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/action_chain_runner/action_chain_runner.py", line 496, in _run_chain
    action_params=action_parameters, context_result=context_result)
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/action_chain_runner/action_chain_runner.py", line 736, in _get_next_action
    parent_context=parent_context)
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/action_chain_runner/action_chain_runner.py", line 796, in _build_liveaction_object
    params=resolved_params)
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/models/utils/action_param_utils.py", line 135, in cast_params
    raise ValueError(msg)
ValueError: Failed to cast value "carlos@XXXXX" (type: str) for parameter "email_to" of type "array": malformed node or string: <_ast.BinOp object at 0x7f3a07b07b00>. Perhaps the value is of an invalid type?

The stack trace from the Action-Chain pin points the actual issue, which is an incorrect data type being provided to the action parameter.

From the action email.send_mail it is defined as an array type.

        "email_to": {
            "description": "Email addresses to send TO.",
            "required": true,
            "type": "array"
        },