StackStorm / st2cd

Content pack for internal build process at StackStorm
Apache License 2.0
7 stars 16 forks source link

Investigate and possibly remove pip+virtualenv from setup_e2e_tests.sh #454

Open blag opened 3 years ago

blag commented 3 years ago

The pip installation and virtualenv creation in setup_e2e_tests.sh might now be superfluous, see here for context (also copied down below).

Wayyy back in the day we used RobotFramework to test ChatOps, but I ripped those out and replaced them with BATS tests, see:

And our current code runs the ChatOps tests using the virtualenv for the chatops_tests pack:

  # Run the tests with Slack
  # Default SLACK_WAIT_FOR_MESSAGES_TIMEOUT value is 120 (seconds)
  run_with_small_timeout:
    action: core.remote
    input:
      hosts: <% ctx().host_ip %>
      timeout: 2400  # seconds
      # Workaround for https://github.com/StackStorm/st2/issues/4676
      # cmd: st2 run --inherit-env tests.test_aliases_with_slack
      # env:
      #   ST2_AUTH_TOKEN: <% ctx().st2_auth_token %>
      #   WEBSOCKET_CLIENT_CA_BUNDLE: <% ctx().websocket %>
      #   SLACK_CHANNEL: <% ctx().slack_channel %>
      #   SLACK_BOT_USERNAME: <% ctx().slack_bot_username %>
      #   SLACK_USER_USERNAME: <% ctx().slack_user_username %>
      #   SLACK_USER_API_TOKEN: <% ctx().slack_user_api_token %>
      cmd: |
        export ST2_AUTH_TOKEN=<% ctx().st2_auth_token %>;
        export WEBSOCKET_CLIENT_CA_BUNDLE=<% ctx().websocket %>;
        export SLACK_CHANNEL=<% ctx().slack_channel %>;
        export SLACK_BOT_USERNAME=<% ctx().slack_bot_username %>;
        export SLACK_USER_USERNAME=<% ctx().slack_user_username %>;
        export SLACK_USER_API_TOKEN=<% ctx().slack_user_api_token %>;
        export SLACK_WAIT_FOR_MESSAGES_TIMEOUT=240;
        . /opt/stackstorm/virtualenvs/chatops_tests/bin/activate;
        python /opt/stackstorm/packs/chatops_tests/actions/test_aliases_with_slack.py

So I think this entire pip+virtualenv is now superfluous and can be removed.

If I remember correctly, we don't use RobotFramework anywhere anymore, so this looks like some cruft that needs cleaning up.

setup_e2e_tests.sh:

# Robotframework requirements
cd st2tests
sudo ${PIP} install --upgrade "pip>=9.0,<9.1"
sudo ${PIP} install --upgrade "virtualenv==15.1.0"

virtualenv --no-download venv
. venv/bin/activate
${PIP} install -r test-requirements.txt

We need to investigate whether or not we can simply remove that.

winem commented 3 years ago

On a first glance, it seems to be safe to remove it. I'll take another look over the long weekend and see if I find anything.

What would be the things to verify and test, if we remove if and want to be sure that it didn't harm? I am not sure if I got the whole picture already.

blag commented 3 years ago

To test it, we would just:

  1. Clone the branch down to st2cicd (I do this in ~/st2cd)
  2. Checkout the branch that we'd like to test (that has those few lines removed from setup_e2e_tests.sh)
  3. Install the st2cd pack with st2 pack install file://$(pwd)
  4. Run at least one end-to-end test, making sure to set chatops=true so that the ChatOps end-to-end tests are also run

If everything passes, then we don't need that code, we merge the st2cd branch on GitHub, and install the st2cd pack from master.

If things break, we will have to investigate why and fix them (or just put the virtualenv back).