StackStorm / st2

StackStorm (aka "IFTTT for Ops") is event-driven automation for auto-remediation, incident responses, troubleshooting, deployments, and more for DevOps and SREs. Includes rules engine, workflow, 160 integration packs with 6000+ actions (see https://exchange.stackstorm.org) and ChatOps. Installer at https://docs.stackstorm.com/install/index.html
https://stackstorm.com/
Apache License 2.0
6.05k stars 745 forks source link

st2 doesn't validate ActionChain task parameters when loading workflow, tasks with missing parameters are "silently" skipped on workflow execution #3938

Open stevemuskiewicz opened 6 years ago

stevemuskiewicz commented 6 years ago

I have an ActionChain workflow with several tasks that are remote-shell-script runners. When creating my yaml metadata for the workflow, I neglected to include the required hosts parameter for the task declaration in the workflow. I would have expected that st2 would have validated the task parameters when loading my pack, but it apparently does not do that.

Even more confusingly, when the workflow executes the task is "skipped" due to the ValidationError exception (I eventually found the traceback buried in one of the st2actionrunner.<pid>.log files) but the workflow continues on executing whatever tasks follow the "skipped" task (it still appears to follow the on-failure task declaration for the task that doesn't run due to the exception). Looking at the execution history in the UI or command line for the workflow, there is no indication anywhere that the task hit this exception, it just isn't shown in the execution list at all.

At the very least, I would expect that the task ValidationError would cause it to show up as "failed" (or maybe some other error state) in the execution history. Instead it's like it isn't even there in the workflow. Also the workflow status after execution ends up being "Succeeded" which seems like it should be "Failed" given that one of the tasks threw an exception.

stevemuskiewicz commented 6 years ago

Poking around a little more in the UI output (now that I know what I'm looking for :-), it does look like clicking on the action output for the workflow and searching for the id of the task that threw the exception does show the exception that occurred. But the only way I found this was drilling down into the output, in the left pane I would have expected to see an entry for the failed task (send_build_notifications) but it's not there

"result": {
        "traceback": "Traceback (most recent call last):
  File \"/opt/stackstorm/runners/action_chain_runner/action_chain_runner.py\", line 506, in _run_chain
    liveaction = self._run_action(liveaction)
  File \"/opt/stackstorm/runners/action_chain_runner/action_chain_runner.py\", line 724, in _run_action
    raise e
ValidationError: u'hosts' is a required property

Failed validating 'required' in schema['properties'][u'hosts']:
    {'additionalProperties': False,
     'description': u'Run build notifier script for completion of the specified product build',
     'properties': {u'bastion_host': {u'description': u'The host SSH connections will be proxied through. Note: This connection is made using the same parameters as the final connection, and is only used in ParamikoSSHRunner.',                                      u'required': False,                                      u'type': [u'string', 'null']},                    u'build_specifier': {u'description': u'Build specifier',                                         u'required': True,                                         u'type': u'string'},                    u'cwd': {u'default': u'/tmp',                             u'description': u'Working directory where the script will be executed in.',                             u'type': u'string'},                    u'dir': {u'default': u'/tmp',                             u'description': u'The working directory where the script will be copied to on the remote host.',                             u'type': u'string'},                    u'env': {u'description': u'Environment variables which will be available to the script(e.g. key1=val1,key2=val2)',                             u'type': [u'object', 'null']},                    u'hosts': {u'description': u'A comma delimited string of a list of hosts where the remote command will be executed.',                               u'required': True,                               u'type': u'string'},                    u'kwarg_op': {u'default': u'--',                                  u'description': u'Operator to use in front of keyword args i.e. \"--\" or \"-\".',                                  u'type': u'string'},                    u'parallel': {u'default': False,                                  u'description': u'Default to parallel execution.',                                  u'immutable': True,                                  u'type': u'boolean'},                    u'passphrase': {u'description': u'Passphrase for the private key, if needed.',                                    u'required': False,                                    u'secret': True,                                    u'type': [u'string', 'null']},                    u'password': {u'description': u'Password used to log in. If not provided, private key from the config file is used.',                                  u'required': False,                                  u'secret': True,                                  u'type': [u'string', 'null']},                    u'port': {u'default': 22,                              u'description': u'SSH port. Note: This parameter is used only in ParamikoSSHRunner.',                              u'required': False,                              u'type': u'integer'},                    u'private_key': {u'description': u'Private key material to log in. Note: This needs to be actual private key data and NOT path.',                                     u'required': False,                                     u'secret': True,                                     u'type': [u'string', 'null']},                    u'product': {u'description': u'Product name',                                 u'required': True,                                 u'type': u'string'},                    u'sudo': {u'default': False,                              u'description': u'The remote command will be executed with sudo.',                              u'type': u'boolean'},                    u'timeout': {u'default': 300,                                 u'description': u'Execution timeout',                                 u'type': u'integer'},                    u'username': {u'description': u'Username used to log-in. If not provided, default username from config is used.',                                  u'required': False,                                  u'type': [u'string', 'null']}},     'title': u'notify_feature_release_builds',     'type': 'object'}On instance[u'hosts']:    {'build_specifier': u'feature/test-bld/13',     u'cwd': u'/tmp',     u'dir': u'/tmp',     u'kwarg_op': u'--',     u'parallel': False,     u'port': 22,     'product': u'platform',     u'sudo': False,     u'timeout': 300}",
        "error": "Failed running task \"send_build_notifications\".. u'hosts' is a required propertyFailed validating 'required' in schema['properties'][u'hosts']:    {'additionalProperties': False,     'description': u'Run build notifier script for completion of the specified product build',     'properties': {u'bastion_host': {u'description': u'The host SSH connections will be proxied through. Note: This connection is made using the same parameters as the final connection, and is only used in ParamikoSSHRunner.',
                                      u'required': False,
                                      u'type': [u'string', 'null']},
                    u'build_specifier': {u'description': u'Build specifier',
                                         u'required': True,
                                         u'type': u'string'},
                    u'cwd': {u'default': u'/tmp',
                             u'description': u'Working directory where the script will be executed in.',
                             u'type': u'string'},
                    u'dir': {u'default': u'/tmp',
                             u'description': u'The working directory where the script will be copied to on the remote host.',
                             u'type': u'string'},
                    u'env': {u'description': u'Environment variables which will be available to the script(e.g. key1=val1,key2=val2)',
                             u'type': [u'object', 'null']},
                    u'hosts': {u'description': u'A comma delimited string of a list of hosts where the remote command will be executed.',
                               u'required': True,
                               u'type': u'string'},
                    u'kwarg_op': {u'default': u'--',
                                  u'description': u'Operator to use in front of keyword args i.e. \"--\" or \"-\".',
                                  u'type': u'string'},
                    u'parallel': {u'default': False,
                                  u'description': u'Default to parallel execution.',
                                  u'immutable': True,
                                  u'type': u'boolean'},
                    u'passphrase': {u'description': u'Passphrase for the private key, if needed.',
                                    u'required': False, 
                                   u'secret': True,
                                    u'type': [u'string', 'null']},
                    u'password': {u'description': u'Password used to log in. If not provided, private key from the config file is used.',
                                  u'required': False,
                                  u'secret': True,
                                  u'type': [u'string', 'null']},
                    u'port': {u'default': 22, 
                             u'description': u'SSH port. Note: This parameter is used only in ParamikoSSHRunner.',
                              u'required': False,
                              u'type': u'integer'},
                    u'private_key': {u'description': u'Private key material to log in. Note: This needs to be actual private key data and NOT path.',
                                     u'required': False,
                                     u'secret': True,
                                     u'type': [u'string', 'null']},
                    u'product': {u'description': u'Product name',
                                 u'required': True,
                                 u'type': u'string'},
                    u'sudo': {u'default': False,
                              u'description': u'The remote command will be executed with sudo.',
                              u'type': u'boolean'},
                    u'timeout': {u'default': 300,
                                 u'description': u'Execution timeout',
                                 u'type': u'integer'},
                    u'username': {u'description': u'Username used to log-in. If not provided, default username from config is used.',
                                  u'required': False,
                                  u'type': [u'string', 'null']}},
     'title': u'notify_feature_release_builds',
     'type': 'object'}

On instance[u'hosts']:
    {'build_specifier': u'feature/test-bld/13',
     u'cwd': u'/tmp',
     u'dir': u'/tmp',
     u'kwarg_op': u'--',
     u'parallel': False,
     u'port': 22,
     'product': u'platform',
     u'sudo': False,
     u'timeout': 300}"
      },