StackStorm / st2ci

New and improved continuous integration actions and workflows
Apache License 2.0
3 stars 9 forks source link

Use the alternatives mechanism of RHEL 8 to set a default Python #163

Closed blag closed 4 years ago

blag commented 4 years ago

RHEL 8 does not set a default version of Python (2 vs. 3). Due to this, there is no python in PATH, so when streamwriter-script.py on RHEL 8, its shebang:

#!/usr/bin/env python

cannot find python, which leads to failures in both the test_quickstart_local_script and test_quickstart_remote_script tests:

test_quickstart_local_script error

```json { "10.0.3.220": { "failed": true, "stderr": "", "return_code": 1, "succeeded": false, "stdout": ". id: 5e6224a102b6f65365877213 action.ref: tests.test_quickstart_local_script_actions parameters: protocol: http token: 4971f3534fe44510b8942acb6d112607 status: failed result_task: test_stdout_local_script_action result: failed: true return_code: 127 stderr: \"/usr/bin/env: \\u2018python\\u2019: No such file or directory\" stdout: '' succeeded: false error: /usr/bin/env: 'python': No such file or directory traceback: None failed_on: test_stdout_local_script_action start_timestamp: Fri, 06 Mar 2020 10:23:29 UTC end_timestamp: Fri, 06 Mar 2020 10:23:30 UTC +--------------------------+---------------------+--------------------------+--------------------------+--------------------------+ | id | status | task | action | start_timestamp | +--------------------------+---------------------+--------------------------+--------------------------+--------------------------+ | 5e6224a1211276a7c1df199c | failed (0s elapsed) | test_stdout_local_script | fixtures.streamwriter- | Fri, 06 Mar 2020 | | | | _action | script-local | 10:23:29 UTC | +--------------------------+---------------------+--------------------------+--------------------------+--------------------------+" } } ```

test_quickstart_remote_script error

```json { "10.0.3.220": { "failed": true, "stderr": "", "return_code": 1, "succeeded": false, "stdout": ".. id: 5e6224a302b6f6536587721c action.ref: tests.test_quickstart_remote_script_actions parameters: protocol: http token: 4971f3534fe44510b8942acb6d112607 status: failed result_task: test_stdout_remote_script_action result: localhost: failed: true return_code: 127 stderr: \"/usr/bin/env: \\u2018python\\u2019: No such file or directory\" stdout: '' succeeded: false start_timestamp: Fri, 06 Mar 2020 10:23:31 UTC end_timestamp: Fri, 06 Mar 2020 10:23:34 UTC +--------------------------+---------------------+--------------------------+--------------------------+--------------------------+ | id | status | task | action | start_timestamp | +--------------------------+---------------------+--------------------------+--------------------------+--------------------------+ | 5e6224a45af60e3eef900a8a | failed (1s elapsed) | test_stdout_remote_scrip | fixtures.streamwriter- | Fri, 06 Mar 2020 | | | | t_action | script-remote | 10:23:32 UTC | +--------------------------+---------------------+--------------------------+--------------------------+--------------------------+" } } ```

While it is possible to specify the exact version of Python to use, neither specification of python2 or python3 covers all possible test cases across all currently supported operating systems: Ubuntu 14, 16, 18, and RHEL 6, 7, and 8. I don't think we offer Python 3 on Ubuntu 14 or RHEL 6 at all, although I could be wrong.

Environment variables are not interpolated into shebang lines, so that won't work either.

PEP 394, which covers how to call Python has this to say about our situation:

In cases where the script is expected to be executed outside virtual environments, developers will need to be aware of the following discrepancies across platforms and installation methods:

  • Older Linux distributions will provide a python command that refers to Python 2, and will likely not provide a python2 command.
  • Some newer Linux distributions will provide a python command that refers to Python 3.
  • Some Linux distributions will not provide a python command at all by default, but will provide a python3 command by default.

Emphasis mine.

There is a post on the RedHat blog that covers Python 2 vs. Python 3. In the "Unversioned Python command" section, it indicates that the "alternatives mechanism" should be used to set the default version of Python for each specific system:

alternatives --set python /usr/bin/python3

This PR adds a task that runs that command on RHEL8 systems so the shebang in the streamwriter-script.py script chooses the system default version of Python.

I will have a companion PR for the st2tests repo that ensures that that script also runs with both Python 2 and Python 3.

arm4b commented 4 years ago

Considering it's st2 pack, can we maybe use the StackStorm shebang instead?

See for example https://github.com/StackStorm-Exchange/stackstorm-ansible/blob/master/actions/ansible.py#L1

#!/opt/stackstorm/st2/bin/python
blag commented 4 years ago

That's an interesting idea - I'll try it out.

blag commented 4 years ago

I closed this because a shebang that uses StackStorm's virtualenv's python solves this problem perfectly. Great idea @armab!