Default Parameters Failing to Cast When Executing From ChatOps

[nmaludy]

ISSUE TYPE

• Bug Report

STACKSTORM VERSION

st2 2.7.2, on Python 2.7

OS / ENVIRONMENT / INSTALL METHOD

os = Red Hat Enterprise Linux Server release 7.5 (Maipo)
install method = puppet-st2

SUMMARY

When executing an action-alias that points to an action which contains a parameter of array, object, or integer type having a default value that reads from the datastore, the alias-execution fails. Executing the same action from the CLI via st2 run works as expected.

STEPS TO REPRODUCE

/opt/stackstorm/packs/default/actions/test_chatops_datastore_default.yaml

---
description: >
  Demonstrates ChatOps failing when reading defaults from datastore
enabled: true
entry_point: workflows/test_chatops_datastore_default.yaml
name: test_chatops_datastore_default
pack: default
parameters:
  kv_array:
    type: array
    default: "{{ st2kv.system.kv_array }}"
    required: true
  kv_integer:
    type: integer
    default: "{{ st2kv.system.kv_integer }}"
    required: true
  kv_number:
    type: number
    default: "{{ st2kv.system.kv_number }}"
    required: true
  kv_object:
    type: object
    default: "{{ st2kv.system.kv_object }}"
    required: true
runner_type: "mistral-v2"

/opt/stackstorm/packs/default/actions/workflows/test_chatops_datastore_default.yaml

version: '2.0'

default.test_chatops_datastore_default:
  type: direct
  input:
    - kv_array
    - kv_integer
    - kv_number
    - kv_object

  output:
    array: "{{ _.kv_array }}"
    integer: "{{ _.kv_integer }}"
    number: "{{ _.kv_number }}"
    object: "{{ _.kv_object }}"

  tasks:
    empty:
      action: std.noop

/opt/stackstorm/packs/default/aliases/test_chatops_datastore_default.yaml

---
name: "test_chatops_datastore_default"
action_ref: "default.test_chatops_datastore_default"
pack: "default"
description: "Demos read failure from datstore"
formats:
    - "test defaults"
    - "test defaults {{ kv_integer }} {{ kv_number }} {{ kv_array }} {{ kv_object }}"
result:
    format: |
        Here's the results of the workflow:
          array: {{ execution.result.array }}
          integer: {{ execution.result.integer }}
          number: {{ execution.result.number }}
          object: {{ execution.result.object }}

Set data in datastore

$ st2 key set kv_array '["a", "b", "c"]'
$ st2 key set kv_integer 7
$ st2 key set kv_number 12.34
$ st2 key set kv_object '{"test": "value"}'
$ st2 key set kv_array '["a", "b", "c"]'

EXPECTED RESULTS

ACTUAL RESULTS

Demo working from CLI

You can see here that i'm passing in no data, all parameters are being read from the datastore, and the action succeeds.

$ st2 run default.test_chatops_datastore_default
.
id: 5b113e2d032fb6c36e746937
action.ref: default.test_chatops_datastore_default
parameters: 
  kv_array:
  - a
  - b
  - c
  kv_integer: 7
  kv_number: 12.34
  kv_object:
    test: value
status: succeeded
start_timestamp: Fri, 01 Jun 2018 12:38:05 UTC
end_timestamp: Fri, 01 Jun 2018 12:38:07 UTC
result: 
  array:
  - a
  - b
  - c
  extra:
    state: SUCCESS
    state_info: null
  integer: 7
  number: 12.34
  object:
    test: value
  tasks:
  - created_at: '2018-06-01 12:38:06'
    id: b0d1b3c0-adbd-4f54-8f2f-5f86b0d6c389
    input: null
    name: empty
    published: {}
    result: null
    state: SUCCESS
    state_info: null
    updated_at: '2018-06-01 12:38:06'
    workflow_execution_id: 8162f038-fcf8-412d-aa0d-a0ea9fa978ee
    workflow_name: default.test_chatops_datastore_default

Demo broken from ChatOps

In this picture you can see my first command test defaults fails because the rendering/casting of the parameters doesn't work. However if i pass in all of the data everything is fine.

Logs from /var/log/st2/st2api.log

2018-06-01 08:40:10,708 140600488785872 ERROR aliasexecution [-] Unable to execute action. Parameter validation failed.
Traceback (most recent call last):
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/st2api/controllers/v1/aliasexecution.py", line 235, in _schedule_execution
    _, action_execution_db = action_service.request(liveaction)
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/st2common/services/action.py", line 158, in request
    liveaction, execution = create_request(liveaction)
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/st2common/services/action.py", line 89, in create_request
    allow_default_none=True)
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/st2common/util/schema/__init__.py", line 294, in validate
    jsonschema.validate(instance=instance, schema=schema, cls=cls, *args, **kwargs)
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/jsonschema/validators.py", line 541, in validate
    cls(schema, *args, **kwargs).validate(instance)
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/jsonschema/validators.py", line 130, in validate
    raise error
ValidationError: u'{{ st2kv.system.kv_number }}' is not of type u'number'

Failed validating u'type' in schema['properties'][u'kv_number']:
    {u'default': u'{{ st2kv.system.kv_number }}',
     u'required': True,
     u'type': u'number'}

On instance[u'kv_number']:
    u'{{ st2kv.system.kv_number }}'
2018-06-01 08:40:10,711 140600488785872 ERROR router [-] Failed to call controller function "post" for operation "st2api.controllers.v1.aliasexecution:action_alias_execution_controller.post": u'{{ st2kv.system.kv_number }}' is not of type u'number'

Failed validating u'type' in schema['properties'][u'kv_number']:
    {u'default': u'{{ st2kv.system.kv_number }}',
     u'required': True,
     u'type': u'number'}

On instance[u'kv_number']:
    u'{{ st2kv.system.kv_number }}'
Traceback (most recent call last):
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/st2common/router.py", line 470, in __call__
    resp = func(**kw)
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/st2api/controllers/v1/aliasexecution.py", line 192, in post
    results = self._post(payload, requester_user, show_secrets, match_multiple=False)
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/st2api/controllers/v1/aliasexecution.py", line 157, in _post
    requester_user=requester_user)
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/st2api/controllers/v1/aliasexecution.py", line 243, in _schedule_execution
    abort(http_client.BAD_REQUEST, str(e))
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/st2common/router.py", line 55, in abort
    raise exc.status_map[status_code](message)
HTTPBadRequest: u'{{ st2kv.system.kv_number }}' is not of type u'number'

Failed validating u'type' in schema['properties'][u'kv_number']:
    {u'default': u'{{ st2kv.system.kv_number }}',
     u'required': True,
     u'type': u'number'}

On instance[u'kv_number']:
    u'{{ st2kv.system.kv_number }}'
2018-06-01 08:40:10,714 140600488785872 INFO logging [-] 1240f333-20df-4866-8fbb-2c035ab8a68a - 400 318 227.747ms (content_length=318,request_id='1240f333-20df-4866-8fbb-2c035ab8a68a',runtime=227.747,remote_addr='127.0.0.1',status=400,method='POST',path='/v1/aliasexecution')

[Kami]

As a temporary workaround, when https://github.com/StackStorm/st2/pull/4155 is merged, you could add explicit int cast in the default value Jinja template.

I believe that should work, but not 100%.

[nmaludy]

Wondering if this affects other executions like rules/webhooks (we don't use those very often, so i haven't tested)?

[nmaludy]

@Kami Ran into this again and tried your suggestion, unfortunately it didn't work. It seems like it never tries to render the Jinja expression in the default when executed from an Alias.

[Kami]

@nmaludy Roger, will look into it and try to reproduce it when I get a chance.