search

StackStorm / ansible-st2

Ansible Roles and Playbooks to deploy StackStorm
https://galaxy.ansible.com/StackStorm/stackstorm/
Apache License 2.0
100 stars 75 forks source link

StackStorm.st2smoketests pack reload fails due to non-ascii character #286

Closed MarvinMuuss closed 3 years ago

MarvinMuuss commented 3 years ago

Hello there,

when I run the ansible playbook reloading the services of st2 fails. This seems to be related to the content of the files (in this case /opt/stackstorm/packs/st2/aliases/actions_list.yaml) which contains a non-ascii character. Running st2ctl reload --register-all on the machine itself runs fine though. I've attached the stacktrace.

Ansible output ``` TASK [StackStorm.st2smoketests : Make sure packs are reloaded] ************************************************************************************************************************* fatal: [172.30.241.95]: FAILED! => {"changed": false, "cmd": ["st2ctl", "reload", "--register-all"], "delta": "0:00:05.209304", "end": "2021-01-14 10:29:29.030379", "msg": "non-zero return code", "rc": 1, "start": "2021-01-14 10:29:23.821075", "stderr": "2021-01-14 10:29:24,703 INFO [-] Connecting to database \"st2\" @ \"127.0.0.1:27017\" as user \"None\". 2021-01-14 10:29:24,707 INFO [-] Successfully connected to database \"st2\" @ \"127.0.0.1:27017\" as user \"None\". 2021-01-14 10:29:25,097 INFO [-] ========================================================= 2021-01-14 10:29:25,097 INFO [-] ############## Registering triggers ##################### 2021-01-14 10:29:25,097 INFO [-] ========================================================= 2021-01-14 10:29:25,188 INFO [-] Registered 0 triggers. 2021-01-14 10:29:25,188 INFO [-] ========================================================= 2021-01-14 10:29:25,188 INFO [-] ############## Registering sensors ###################### 2021-01-14 10:29:25,188 INFO [-] ========================================================= 2021-01-14 10:29:25,211 INFO [-] Registered 1 sensors. 2021-01-14 10:29:25,211 INFO [-] ========================================================= 2021-01-14 10:29:25,211 INFO [-] ############## Registering runners ###################### 2021-01-14 10:29:25,211 INFO [-] ========================================================= 2021-01-14 10:29:26,180 INFO [-] Registered 14 runners. 2021-01-14 10:29:26,180 INFO [-] ========================================================= 2021-01-14 10:29:26,180 INFO [-] ############## Registering actions ###################### 2021-01-14 10:29:26,180 INFO [-] ========================================================= 2021-01-14 10:29:28,205 INFO [-] Registered 81 actions. 2021-01-14 10:29:28,205 INFO [-] ========================================================= 2021-01-14 10:29:28,205 INFO [-] ############## Registering rules ######################## 2021-01-14 10:29:28,205 INFO [-] ========================================================= 2021-01-14 10:29:28,234 INFO [-] Registered 1 rules. 2021-01-14 10:29:28,234 INFO [-] ========================================================= 2021-01-14 10:29:28,234 INFO [-] ############## Registering aliases ###################### 2021-01-14 10:29:28,234 INFO [-] ========================================================= 2021-01-14 10:29:28,235 ERROR [-] Failed loading content from /opt/stackstorm/packs/st2/aliases/actions_list.yaml. Traceback (most recent call last): File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/content/loader.py\", line 248, in _load return parser_func(fd) File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/__init__.py\", line 162, in safe_load return load(stream, SafeLoader) File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/__init__.py\", line 112, in load loader = Loader(stream) File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/loader.py\", line 34, in __init__ Reader.__init__(self, stream) File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/reader.py\", line 85, in __init__ self.determine_encoding() File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/reader.py\", line 124, in determine_encoding self.update_raw() File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/reader.py\", line 178, in update_raw data = self.stream.read(size) File \"/opt/stackstorm/st2/lib64/python3.6/encodings/ascii.py\", line 26, in decode return codecs.ascii_decode(input, self.errors)[0] UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 439: ordinal not in range(128) Traceback (most recent call last): File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/bootstrap/aliasesregistrar.py\", line 175, in _register_aliases_from_pack self._register_action_alias(pack, alias) File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/bootstrap/aliasesregistrar.py\", line 146, in _register_action_alias action_alias=action_alias) File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/bootstrap/aliasesregistrar.py\", line 117, in _get_action_alias_db content = self._meta_loader.load(action_alias) File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/content/loader.py\", line 236, in load result = self._load(PARSER_FUNCS[file_ext], file_path) File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/content/loader.py\", line 248, in _load return parser_func(fd) File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/__init__.py\", line 162, in safe_load return load(stream, SafeLoader) File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/__init__.py\", line 112, in load loader = Loader(stream) File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/loader.py\", line 34, in __init__ Reader.__init__(self, stream) File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/reader.py\", line 85, in __init__ self.determine_encoding() File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/reader.py\", line 124, in determine_encoding self.update_raw() File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/reader.py\", line 178, in update_raw data = self.stream.read(size) File \"/opt/stackstorm/st2/lib64/python3.6/encodings/ascii.py\", line 26, in decode return codecs.ascii_decode(input, self.errors)[0] UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 439: ordinal not in range(128) During handling of the above exception, another exception occurred: Traceback (most recent call last): File \"/bin/st2-register-content\", line 22, in sys.exit(content_loader.main(sys.argv[1:])) File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/content/bootstrap.py\", line 408, in main register_content() File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/content/bootstrap.py\", line 360, in register_content register_aliases() File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/content/bootstrap.py\", line 283, in register_aliases raise e File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/content/bootstrap.py\", line 279, in register_aliases fail_on_failure=fail_on_failure File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/bootstrap/aliasesregistrar.py\", line 205, in register_aliases result = registrar.register_from_packs(base_dirs=packs_base_paths) File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/bootstrap/aliasesregistrar.py\", line 67, in register_from_packs raise e File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/bootstrap/aliasesregistrar.py\", line 63, in register_from_packs count = self._register_aliases_from_pack(pack=pack, aliases=aliases) File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/bootstrap/aliasesregistrar.py\", line 180, in _register_aliases_from_pack raise ValueError(msg) ValueError: Failed to register alias \"/opt/stackstorm/packs/st2/aliases/actions_list.yaml\" from pack \"st2\": 'ascii' codec can't decode byte 0xe2 in position 439: ordinal not in range(128)", "stderr_lines": ["2021-01-14 10:29:24,703 INFO [-] Connecting to database \"st2\" @ \"127.0.0.1:27017\" as user \"None\".", "2021-01-14 10:29:24,707 INFO [-] Successfully connected to database \"st2\" @ \"127.0.0.1:27017\" as user \"None\".", "2021-01-14 10:29:25,097 INFO [-] =========================================================", "2021-01-14 10:29:25,097 INFO [-] ############## Registering triggers #####################", "2021-01-14 10:29:25,097 INFO [-] =========================================================", "2021-01-14 10:29:25,188 INFO [-] Registered 0 triggers.", "2021-01-14 10:29:25,188 INFO [-] =========================================================", "2021-01-14 10:29:25,188 INFO [-] ############## Registering sensors ######################", "2021-01-14 10:29:25,188 INFO [-] =========================================================", "2021-01-14 10:29:25,211 INFO [-] Registered 1 sensors.", "2021-01-14 10:29:25,211 INFO [-] =========================================================", "2021-01-14 10:29:25,211 INFO [-] ############## Registering runners ######################", "2021-01-14 10:29:25,211 INFO [-] =========================================================", "2021-01-14 10:29:26,180 INFO [-] Registered 14 runners.", "2021-01-14 10:29:26,180 INFO [-] =========================================================", "2021-01-14 10:29:26,180 INFO [-] ############## Registering actions ######################", "2021-01-14 10:29:26,180 INFO [-] =========================================================", "2021-01-14 10:29:28,205 INFO [-] Registered 81 actions.", "2021-01-14 10:29:28,205 INFO [-] =========================================================", "2021-01-14 10:29:28,205 INFO [-] ############## Registering rules ########################", "2021-01-14 10:29:28,205 INFO [-] =========================================================", "2021-01-14 10:29:28,234 INFO [-] Registered 1 rules.", "2021-01-14 10:29:28,234 INFO [-] =========================================================", "2021-01-14 10:29:28,234 INFO [-] ############## Registering aliases ######################", "2021-01-14 10:29:28,234 INFO [-] =========================================================", "2021-01-14 10:29:28,235 ERROR [-] Failed loading content from /opt/stackstorm/packs/st2/aliases/actions_list.yaml.", "Traceback (most recent call last):", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/content/loader.py\", line 248, in _load", " return parser_func(fd)", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/__init__.py\", line 162, in safe_load", " return load(stream, SafeLoader)", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/__init__.py\", line 112, in load", " loader = Loader(stream)", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/loader.py\", line 34, in __init__", " Reader.__init__(self, stream)", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/reader.py\", line 85, in __init__", " self.determine_encoding()", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/reader.py\", line 124, in determine_encoding", " self.update_raw()", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/reader.py\", line 178, in update_raw", " data = self.stream.read(size)", " File \"/opt/stackstorm/st2/lib64/python3.6/encodings/ascii.py\", line 26, in decode", " return codecs.ascii_decode(input, self.errors)[0]", "UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 439: ordinal not in range(128)", "Traceback (most recent call last):", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/bootstrap/aliasesregistrar.py\", line 175, in _register_aliases_from_pack", " self._register_action_alias(pack, alias)", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/bootstrap/aliasesregistrar.py\", line 146, in _register_action_alias", " action_alias=action_alias)", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/bootstrap/aliasesregistrar.py\", line 117, in _get_action_alias_db", " content = self._meta_loader.load(action_alias)", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/content/loader.py\", line 236, in load", " result = self._load(PARSER_FUNCS[file_ext], file_path)", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/content/loader.py\", line 248, in _load", " return parser_func(fd)", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/__init__.py\", line 162, in safe_load", " return load(stream, SafeLoader)", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/__init__.py\", line 112, in load", " loader = Loader(stream)", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/loader.py\", line 34, in __init__", " Reader.__init__(self, stream)", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/reader.py\", line 85, in __init__", " self.determine_encoding()", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/reader.py\", line 124, in determine_encoding", " self.update_raw()", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/yaml/reader.py\", line 178, in update_raw", " data = self.stream.read(size)", " File \"/opt/stackstorm/st2/lib64/python3.6/encodings/ascii.py\", line 26, in decode", " return codecs.ascii_decode(input, self.errors)[0]", "UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 439: ordinal not in range(128)", "", "During handling of the above exception, another exception occurred:", "", "Traceback (most recent call last):", " File \"/bin/st2-register-content\", line 22, in ", " sys.exit(content_loader.main(sys.argv[1:]))", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/content/bootstrap.py\", line 408, in main", " register_content()", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/content/bootstrap.py\", line 360, in register_content", " register_aliases()", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/content/bootstrap.py\", line 283, in register_aliases", " raise e", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/content/bootstrap.py\", line 279, in register_aliases", " fail_on_failure=fail_on_failure", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/bootstrap/aliasesregistrar.py\", line 205, in register_aliases", " result = registrar.register_from_packs(base_dirs=packs_base_paths)", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/bootstrap/aliasesregistrar.py\", line 67, in register_from_packs", " raise e", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/bootstrap/aliasesregistrar.py\", line 63, in register_from_packs", " count = self._register_aliases_from_pack(pack=pack, aliases=aliases)", " File \"/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/bootstrap/aliasesregistrar.py\", line 180, in _register_aliases_from_pack", " raise ValueError(msg)", "ValueError: Failed to register alias \"/opt/stackstorm/packs/st2/aliases/actions_list.yaml\" from pack \"st2\": 'ascii' codec can't decode byte 0xe2 in position 439: ordinal not in range(128)"], "stdout": "Registering content...[flags = --config-file /etc/st2/st2.conf --register-all] ##### st2 components status ##### st2actionrunner PID: 18661 st2actionrunner PID: 18663 st2actionrunner PID: 18665 st2actionrunner PID: 18667 st2actionrunner PID: 18669 st2actionrunner PID: 18671 st2actionrunner PID: 18673 st2actionrunner PID: 18675 st2actionrunner PID: 18677 st2actionrunner PID: 18679 st2api PID: 87707 st2api PID: 87772 st2stream PID: 87833 st2stream PID: 87852 st2auth PID: 19594 st2auth PID: 19665 st2garbagecollector PID: 18848 st2notifier PID: 18971 st2resultstracker is not running. st2rulesengine PID: 19095 st2sensorcontainer PID: 19219 st2chatops is not running. st2timersengine PID: 19964 st2workflowengine PID: 19843 st2scheduler PID: 19720", "stdout_lines": ["Registering content...[flags = --config-file /etc/st2/st2.conf --register-all]", "##### st2 components status #####", "st2actionrunner PID: 18661", "st2actionrunner PID: 18663", "st2actionrunner PID: 18665", "st2actionrunner PID: 18667", "st2actionrunner PID: 18669", "st2actionrunner PID: 18671", "st2actionrunner PID: 18673", "st2actionrunner PID: 18675", "st2actionrunner PID: 18677", "st2actionrunner PID: 18679", "st2api PID: 87707", "st2api PID: 87772", "st2stream PID: 87833", "st2stream PID: 87852", "st2auth PID: 19594", "st2auth PID: 19665", "st2garbagecollector PID: 18848", "st2notifier PID: 18971", "st2resultstracker is not running.", "st2rulesengine PID: 19095", "st2sensorcontainer PID: 19219", "st2chatops is not running.", "st2timersengine PID: 19964", "st2workflowengine PID: 19843", "st2scheduler PID: 19720"]} ```

I work on fedora 32, python 3.8.6 with ansible-playbook in version 2.9.16. The target system is a CentOs 8.2, the virtual environment runs with python 3.6.8 and this is the content of pip freeze

Pip freeze ``` amqp==2.5.2 APScheduler==3.6.3 argcomplete==1.12.1 Babel==2.8.0 bcrypt==3.1.7 beautifulsoup4==4.9.3 certifi==2020.6.20 cffi==1.14.3 chardet==3.0.4 click==7.1.2 colorama==0.4.4 cryptography==2.8 debtcollector==2.2.0 decorator==4.4.2 dnspython==1.16.0 eventlet==0.25.1 fasteners==0.15 flex==6.14.0 futurist==2.3.0 gitdb2==2.0.6 GitPython==2.1.15 greenlet==0.4.15 gunicorn==19.9.0 httplib2==0.18.1 idna==2.10 importlib-metadata==2.0.0 importlib-resources==3.0.0 ipaddr==2.2.0 iso8601==0.1.13 Jinja2==2.10.3 jsonpath-rw==1.4.0 jsonpointer==2.0 jsonschema==2.6.0 kazoo==2.8.0 kombu==4.6.6 linecache2==1.0.0 lockfile==0.12.2 logshipper==0.1.1.dev111 MarkupSafe==1.1.1 mock==2.0.0 mongoengine==0.18.2 monotonic==1.5 more-itertools==5.0.0 msgpack==1.0.0 netaddr==0.8.0 netifaces==0.10.9 networkx==1.11 nose==1.3.7 nose-parallel==0.3.1 nose-timer==0.7.5 ntlm-auth==1.5.0 orquesta==1.2.0 oslo.config==1.12.1 oslo.i18n==5.0.1 oslo.serialization==4.0.1 oslo.utils==3.37.0 paramiko==2.7.1 passlib==1.7.1 pbr==5.5.0 pika==1.1.0 ply==3.11 prettytable==1.0.1 prompt-toolkit==1.0.15 psutil==5.6.6 pycparser==2.20 pyinotify==0.9.6 pymongo==3.10.0 PyNaCl==1.4.0 pyOpenSSL==19.1.0 pyparsing==2.4.7 pyrabbit==1.1.0 python-dateutil==2.8.0 python-editor==1.0.4 python-gnupg==0.4.5 python-json-logger==2.0.1 python-statsd==2.1.0 pytz==2019.1 pywinrm==0.3.0 PyYAML==5.1.2 RandomWords==0.3.0 rednose==1.3.0 repoze.lru==0.7 requests==2.23.0 requests-ntlm==1.1.0 retrying==1.3.3 rfc3987==1.3.8 Routes==2.4.1 semver==2.9.0 simplejson==3.17.2 six==1.13.0 smmap==3.0.4 smmap2==3.0.1 soupsieve==2.0.1 sseclient-py==1.7 st2==3.3.0 st2-auth-backend-flat-file==0.1.0 st2-auth-backend-pam==0.1.0 st2actions==3.3.0 st2api==3.3.0 st2auth==3.3.0 st2client==3.3.0 st2common==3.3.0 st2debug==3.3.0 st2exporter==3.3.0 st2reactor==3.3.0 st2stream==3.3.0 st2tests==3.3.0 stackstorm-runner-action-chain==3.3.0 stackstorm-runner-announcement==3.3.0 stackstorm-runner-http==3.3.0 stackstorm-runner-inquirer==3.3.0 stackstorm-runner-local==3.3.0 stackstorm-runner-noop==3.3.0 stackstorm-runner-orquesta==3.3.0 stackstorm-runner-python==3.3.0 stackstorm-runner-remote==3.3.0 stackstorm-runner-winrm==3.3.0 stevedore==1.30.1 strict-rfc3339==0.7 tenacity==6.2.0 termstyle==0.1.11 tooz==1.66.1 traceback2==1.4.0 tzlocal==2.1 ujson==1.35 unittest2==1.1.0 urllib3==1.25.10 validate-email==1.3 vine==1.3.0 virtualenv==16.6.0 voluptuous==0.12.0 waitress==1.4.4 wcwidth==0.2.5 WebOb==1.8.5 WebTest==2.0.35 wrapt==1.12.1 xmltodict==0.12.0 yaql==1.1.3 zake==0.2.2 zipp==1.0.0 ```

If I've missed any information please let me know. Thanks for your support.

Best regards Marvin

arm4b commented 3 years ago

Thanks for the detailed report!

This is not related to any deployment method, but to StackStorm platform itself.

We've fixed this specific issue in the https://github.com/StackStorm/st2/pull/5106 (thanks to @blag) and the fix will be shipped in the upcoming st2 v3.4.0 release (~February). I'm closing this issue as a Duplicate of https://github.com/StackStorm/st2/pull/5106.

You can do the following to workaround it until the release:

Hope that helps.