search

StackStorm / st2

StackStorm (aka "IFTTT for Ops") is event-driven automation for auto-remediation, incident responses, troubleshooting, deployments, and more for DevOps and SREs. Includes rules engine, workflow, 160 integration packs with 6000+ actions (see https://exchange.stackstorm.org) and ChatOps. Installer at https://docs.stackstorm.com/install/index.html
https://stackstorm.com/
Apache License 2.0
6.08k stars 749 forks source link

Unable to run an action implemented by python3 #4280

Closed AutoJunjie closed 6 years ago

AutoJunjie commented 6 years ago
SUMMARY

Unable to run an action implemented by python3.6

ISSUE TYPE
STACKSTORM VERSION

st2 --version st2 2.8.1, on Python 2.7.5

OS / ENVIRONMENT / INSTALL METHOD

CentOS7/python3.6.3/Installde by using: curl -sSL https://stackstorm.com/packages/install.sh | bash -s -- --user=st2admin --password='Ch@ngeMe'

STEPS TO REPRODUCE
from __future__ import absolute_import
import sys
print(sys.path)

Traceback:

Traceback (most recent call last):
  File "/opt/stackstorm/runners/python_runner/python_runner/python_action_wrapper.py", line 21, in <module>
    import traceback
  File "/usr/local/lib/python3.6/traceback.py", line 5, in <module>
    import linecache
  File "/opt/stackstorm/virtualenvs/nornir_scripts/lib/python3.6/linecache.py", line 11, in <module>
    import tokenize
  File "/opt/stackstorm/virtualenvs/nornir_scripts/lib/python3.6/tokenize.py", line 33, in <module>
    import re
  File "/opt/stackstorm/virtualenvs/nornir_scripts/lib/python3.6/re.py", line 142, in <module>
    class RegexFlag(enum.IntFlag):
AttributeError: module 'enum' has no attribute 'IntFlag'

Action runner service logs:

[jhuang@chcgilperfops03 st2]$ cat st2actionrunner.24688.log
2018-07-31 03:15:17,573 140698490962320 INFO __init__ [-] Connecting to database "st2" @ "127.0.0.1:27017" as user "stackstorm".
2018-07-31 03:15:17,635 140698490962320 INFO __init__ [-] Successfully connected to database "st2" @ "127.0.0.1:27017" as user "stackstorm".
2018-07-31 03:15:18,005 140698490962320 INFO actionrunner [-] (PID=24688) Worker started.
2018-07-31 03:15:18,006 140698490962320 INFO consumers [-] Starting ActionExecutionScheduler...
2018-07-31 03:15:18,006 140698490962320 INFO consumers [-] Starting ActionExecutionDispatcher...
2018-07-31 03:15:18,045 140698312063248 INFO mixins [-] Connected to amqp://guest:**@127.0.0.1:5672//
2018-07-31 03:15:18,045 140698312062608 INFO mixins [-] Connected to amqp://guest:**@127.0.0.1:5672//
2018-07-31 03:16:44,121 140698312062768 AUDIT action [-] The status of action execution is changed from requested to scheduled. <LiveAction.id=5b5fd49ca4499d613f0d38e6, ActionExecution.id=5b5fd49ca4499d613f0d38e7> (liveaction_db={'status': 'scheduled', 'runner_info': {}, 'workflow_execution': None, 'parameters': {u'debug': True, u'log_level': u'DEBUG'}, 'action_is_workflow': False, 'start_timestamp': '2018-07-31 03:16:44.031922+00:00', 'callback': {}, 'task_execution': None, 'notify': None, 'result': {}, 'context': {u'trace_context': {}, u'user': u'st2admin', u'pack': u'nornir_scirpts'}, 'action': u'nornir_scirpts.print_syspath', 'id': '5b5fd49ca4499d613f0d38e6', 'end_timestamp': None},action_execution_db={'status': u'scheduled', 'start_timestamp': '2018-07-31 03:16:44.031922+00:00', 'web_url': u'https://chcgilperfops03.gogoeng.prod/#/history/5b5fd49ca4499d613f0d38e7/general', 'log': [{u'status': u'requested', u'timestamp': datetime.datetime(2018, 7, 31, 3, 16, 44, 42000, tzinfo=<bson.tz_util.FixedOffset object at 0x7ff6e0b92fd0>)}, {u'status': u'scheduled', u'timestamp': datetime.datetime(2018, 7, 31, 3, 16, 44, 109000, tzinfo=<bson.tz_util.FixedOffset object at 0x7ff6e0b92fd0>)}], 'parameters': {u'debug': True, u'log_level': u'DEBUG'}, 'trigger_type': {}, 'runner': {u'runner_module': u'python_runner', u'uid': u'runner_type:python-script', u'runner_package': u'python_runner', u'description': u'A runner for launching python actions.', u'enabled': True, u'runner_parameters': {u'debug': {u'default': False, u'required': False, u'type': u'boolean', u'description': u'Enable runner debug mode.'}, u'content_version': {u'required': False, u'type': u'string', u'description': u'Git revision of the pack content to use for this action execution (git commit sha / tag / branch). Only applies to packs which are git repositories.'}, u'log_level': {u'default': u'DEBUG', u'enum': [u'AUDIT', u'CRITICAL', u'ERROR', u'WARNING', u'INFO', u'DEBUG'], u'type': u'string', u'description': u'Default log level for Python runner actions.'}, u'env': {u'type': u'object', u'description': u'Environment variables which will be available to the script.'}, u'timeout': {u'default': 600, u'type': u'integer', u'description': u"Action timeout in seconds. Action will get killed if it doesn't finish in timeout seconds."}}, u'id': u'5b5bbc79a4499d5c24c3553d', u'name': u'python-script'}, 'trigger_instance': {}, 'rule': {}, 'children': [], 'task_execution': None, 'trigger': {}, 'parent': None, 'result': {}, 'context': {u'trace_context': {}, u'user': u'st2admin', u'pack': u'nornir_scirpts'}, 'workflow_execution': None, 'action': {u'description': u'python_script', u'parameters': {}, u'tags': [], u'enabled': True, u'name': u'print_syspath', u'entry_point': u'syspath.py', u'notify': {}, u'uid': u'action:nornir_scirpts:print_syspath', u'pack': u'nornir_scirpts', u'ref': u'nornir_scirpts.print_syspath', u'id': u'5b5f90d7a4499d0dc6523a3e', u'runner_type': u'python-script'}, 'liveaction': {u'runner_info': {}, u'parameters': {u'debug': True, u'log_level': u'DEBUG'}, u'action_is_workflow': False, u'callback': {}, u'action': u'nornir_scirpts.print_syspath', u'id': u'5b5fd49ca4499d613f0d38e6'}, 'id': '5b5fd49ca4499d613f0d38e7', 'end_timestamp': None})
2018-07-31 03:16:44,150 140698312062768 INFO action [-] The status of action execution is changed from requested to scheduled. <LiveAction.id=5b5fd49ca4499d613f0d38e6, ActionExecution.id=5b5fd49ca4499d613f0d38e7>
2018-07-31 03:16:44,183 140698312062768 AUDIT worker [-] Launching action execution. (liveaction_db={'status': 'running', 'runner_info': {'hostname': 'chcgilperfops03.gogoeng.prod', 'pid': 24688}, 'workflow_execution': None, 'parameters': {u'debug': True, u'log_level': u'DEBUG'}, 'action_is_workflow': False, 'start_timestamp': '2018-07-31 03:16:44.031922+00:00', 'callback': {}, 'task_execution': None, 'notify': None, 'result': {}, 'context': {u'trace_context': {}, u'user': u'st2admin', u'pack': u'nornir_scirpts'}, 'action': u'nornir_scirpts.print_syspath', 'id': '5b5fd49ca4499d613f0d38e6', 'end_timestamp': None},action_execution_db={'status': u'running', 'start_timestamp': '2018-07-31 03:16:44.031922+00:00', 'web_url': u'https://chcgilperfops03.gogoeng.prod/#/history/5b5fd49ca4499d613f0d38e7/general', 'log': [{u'status': u'requested', u'timestamp': datetime.datetime(2018, 7, 31, 3, 16, 44, 42000, tzinfo=<bson.tz_util.FixedOffset object at 0x7ff6e0bb3410>)}, {u'status': u'scheduled', u'timestamp': datetime.datetime(2018, 7, 31, 3, 16, 44, 109000, tzinfo=<bson.tz_util.FixedOffset object at 0x7ff6e0bb3410>)}, {u'status': u'running', u'timestamp': datetime.datetime(2018, 7, 31, 3, 16, 44, 173000, tzinfo=<bson.tz_util.FixedOffset object at 0x7ff6e0bb3410>)}], 'parameters': {u'debug': True, u'log_level': u'DEBUG'}, 'trigger_type': {}, 'runner': {u'runner_module': u'python_runner', u'uid': u'runner_type:python-script', u'runner_package': u'python_runner', u'description': u'A runner for launching python actions.', u'enabled': True, u'runner_parameters': {u'debug': {u'default': False, u'required': False, u'type': u'boolean', u'description': u'Enable runner debug mode.'}, u'content_version': {u'required': False, u'type': u'string', u'description': u'Git revision of the pack content to use for this action execution (git commit sha / tag / branch). Only applies to packs which are git repositories.'}, u'log_level': {u'default': u'DEBUG', u'enum': [u'AUDIT', u'CRITICAL', u'ERROR', u'WARNING', u'INFO', u'DEBUG'], u'type': u'string', u'description': u'Default log level for Python runner actions.'}, u'env': {u'type': u'object', u'description': u'Environment variables which will be available to the script.'}, u'timeout': {u'default': 600, u'type': u'integer', u'description': u"Action timeout in seconds. Action will get killed if it doesn't finish in timeout seconds."}}, u'id': u'5b5bbc79a4499d5c24c3553d', u'name': u'python-script'}, 'trigger_instance': {}, 'rule': {}, 'children': [], 'task_execution': None, 'trigger': {}, 'parent': None, 'result': {}, 'context': {u'trace_context': {}, u'user': u'st2admin', u'pack': u'nornir_scirpts'}, 'workflow_execution': None, 'action': {u'description': u'python_script', u'parameters': {}, u'tags': [], u'enabled': True, u'name': u'print_syspath', u'entry_point': u'syspath.py', u'notify': {}, u'uid': u'action:nornir_scirpts:print_syspath', u'pack': u'nornir_scirpts', u'ref': u'nornir_scirpts.print_syspath', u'id': u'5b5f90d7a4499d0dc6523a3e', u'runner_type': u'python-script'}, 'liveaction': {u'runner_info': {u'hostname': u'chcgilperfops03.gogoeng.prod', u'pid': 24688}, u'parameters': {u'debug': True, u'log_level': u'DEBUG'}, u'action_is_workflow': False, u'callback': {}, u'action': u'nornir_scirpts.print_syspath', u'id': u'5b5fd49ca4499d613f0d38e6'}, 'id': '5b5fd49ca4499d613f0d38e7', 'end_timestamp': None})
2018-07-31 03:16:44,218 140698312062768 INFO worker [-] Dispatched {~}action_execution: 5b5fd49ca4499d613f0d38e7 / {~}live_action: 5b5fd49ca4499d613f0d38e6 with "running" status.
2018-07-31 03:16:44,224 140698312062768 INFO base [-] Dispatching Action to a runner (runner_type_db={'runner_module': u'python_runner', 'uid': u'runner_type:python-script', 'name': u'python-script', 'runner_package': u'python_runner', 'enabled': True, 'query_module': None, 'runner_parameters': {u'debug': {u'default': False, u'required': False, u'type': u'boolean', u'description': u'Enable runner debug mode.'}, u'content_version': {u'required': False, u'type': u'string', u'description': u'Git revision of the pack content to use for this action execution (git commit sha / tag / branch). Only applies to packs which are git repositories.'}, u'log_level': {u'default': u'DEBUG', u'enum': [u'AUDIT', u'CRITICAL', u'ERROR', u'WARNING', u'INFO', u'DEBUG'], u'type': u'string', u'description': u'Default log level for Python runner actions.'}, u'env': {u'type': u'object', u'description': u'Environment variables which will be available to the script.'}, u'timeout': {u'default': 600, u'type': u'integer', u'description': u"Action timeout in seconds. Action will get killed if it doesn't finish in timeout seconds."}}, 'id': '5b5bbc79a4499d5c24c3553d', 'description': u'A runner for launching python actions.'},liveaction_db={'status': 'running', 'runner_info': {'hostname': 'chcgilperfops03.gogoeng.prod', 'pid': 24688}, 'workflow_execution': None, 'parameters': {u'debug': True, u'log_level': u'DEBUG'}, 'action_is_workflow': False, 'start_timestamp': '2018-07-31 03:16:44.031922+00:00', 'callback': {}, 'task_execution': None, 'notify': None, 'result': {}, 'context': {u'trace_context': {}, u'user': u'st2admin', u'pack': u'nornir_scirpts'}, 'action': u'nornir_scirpts.print_syspath', 'id': '5b5fd49ca4499d613f0d38e6', 'end_timestamp': None})
2018-07-31 03:16:44,243 140698312062768 INFO loader [-] Loading runner module from "/opt/stackstorm/runners/python_runner/python_runner/python_runner.py".
2018-07-31 03:16:44,254 140698312062768 AUDIT access [-] Access granted to "st2admin" with the token set to expire at "2018-08-01T03:16:44.253477Z". (username=u'st2admin',token_expiration='2018-08-01T03:16:44.253477Z')
2018-07-31 03:16:44,258 140698312062768 WARNING api [-] "auth.api_url" configuration option is not configured
2018-07-31 03:16:44,258 140698312062768 WARNING api [-] "auth.api_url" configuration option is not configured
2018-07-31 03:16:44,358 140698312062768 WARNING python_runner [-] Failed to de-serialize result "None": expected string or buffer
2018-07-31 03:16:44,389 140698312062768 AUDIT base [-] Liveaction completed (liveaction_db={'status': 'failed', 'runner_info': {u'hostname': u'chcgilperfops03.gogoeng.prod', u'pid': 24688}, 'workflow_execution': None, 'parameters': {u'debug': True, u'log_level': u'DEBUG'}, 'action_is_workflow': False, 'start_timestamp': '2018-07-31 03:16:44.031922+00:00', 'callback': {}, 'task_execution': None, 'notify': None, 'result': {'stdout': '', 'exit_code': 1, 'stderr': u'Traceback (most recent call last):\n  File "/opt/stackstorm/runners/python_runner/python_runner/python_action_wrapper.py", line 21, in <module>\n    import traceback\n  File "/usr/local/lib/python3.6/traceback.py", line 5, in <module>\n    import linecache\n  File "/opt/stackstorm/virtualenvs/nornir_scirpts/lib/python3.6/linecache.py", line 11, in <module>\n    import tokenize\n  File "/opt/stackstorm/virtualenvs/nornir_scirpts/lib/python3.6/tokenize.py", line 33, in <module>\n    import re\n  File "/opt/stackstorm/virtualenvs/nornir_scirpts/lib/python3.6/re.py", line 142, in <module>\n    class RegexFlag(enum.IntFlag):\nAttributeError: module \'enum\' has no attribute \'IntFlag\'\n', 'result': 'None'}, 'context': {u'trace_context': {}, u'user': u'st2admin', u'pack': u'nornir_scirpts'}, 'action': u'nornir_scirpts.print_syspath', 'id': '5b5fd49ca4499d613f0d38e6', 'end_timestamp': '2018-07-31 03:16:44.361734+00:00'})

Switch to virtual environment, enum.IntFlag is true:

(nornir_scripts) [xxx@xxxxxxx]$ python
Python 3.6.3 (default, May 15 2018, 19:56:00) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-16)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import enum
>>> dir(enum)     
['DynamicClassAttribute', 'Enum', 'EnumMeta', 'Flag', 'IntEnum', 'IntFlag', 'MappingProxyType', 'OrderedDict', '_EnumDict', '__all__', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', '_auto_null', '_decompose', '_high_bit', '_is_descriptor', '_is_dunder', '_is_sunder', '_make_class_unpicklable', '_or_', '_power_of_two', '_reduce_ex_by_name', 'auto', 'reduce', 'sys', 'unique']
>>>
EXPECTED RESULTS

expect printing the sys path

ACTUAL RESULTS
AttributeError: module 'enum' has no attribute 'IntFlag'
LindsayHill commented 6 years ago

Fixed by #4297