Selenium testing in CI

[rouilj]

The Roundup has a medium sized test suite. This includes running against a live server (see test/test_liveserver.py). It would be good to get the tests @BharathKanama is working on into the Roundup suite. We have CI (continuous integration) testing via GitHub actions.

Since the selenium tests could run in CI or on a desktop, run headless chrome if the DISPLAY environment variable is not defined. If DISPLAY is set (even if it is empty), run the browser in "headfull"?? mode. The latter is to be used when running on the desktop.

If the tests won't pass in headless mode for some reason, one of the references shows how to start Xfb as a dummy X server for chrome to use in CI. But running headless would be better.

If there is a way to run in headfull mode in CI and capture the results of the run, that would be interesting info to link to. Implementing is out of scope for this ticket.

It looks like the github action workflow in .github/workflows/ci-test will need to be changed to load chrome (and maybe chrome-driver) using apt and 'selenium' using pip. According to https://selenium-python.readthedocs.io/installation.html newer versions should include https://www.selenium.dev/documentation/selenium_manager/ which is supposed to handle installation of the chrome or other browser driver. From today's standup discusion I have the impression it's not working and the driver has to be explicitly installed??

The selenium tests should be guarded using an @skip_selenium decorator. See how skip_requests or skip_brotli are set up in test_liveserver. This will allow the testsuite to run in an environment where selenium is not loaded.

Create a class TestSelenium that inherits from WsgiSetup that is decorated with @skip_selenium.

References:

https://dev.to/delrayo/executing-selenium-test-with-pythonpytest-using-github-actions-317c

The first answer at: https://stackoverflow.com/questions/62364711/how-can-i-get-selenium-tests-to-run-in-python-with-github-actions looks at how selenium sets up their tests.

https://docs.dev-tools.ai/blog/running-ui-tests-in-github-actions

https://dev.to/pratyushsingh07/integrating-github-actions-with-selenium-cp0 (note I am not a fan of snaps which this answer uses to install chrome)

[rouilj]

In the .github/workflows directory you might want to turn off all of the existing workflows by renaming them with a .disabled extension.

Then copy ci-test to ci-selenium and edit it to make CI fast so you can get a quick turnaround. A full CI run takes about 30 minutes across 5 versions of python. Much too long for developing this test in CI. Once all the moving parts are straightened out, we can merge them into the main ci-test suite.

To speed things up:

• in the jobs/test/strategy/matrix comment out everything except 3.11.
• change os to just [ubuntu-latest] and remove the experimental/include/exclude sections.
• remove all of the steps except:

    - name: Checkout source
    - name: Set Up Python ${{ matrix.python-version }}
    - name: Install build tools - setuptools
    - name: Display Python and key module versions
    - name: Update pip
    - name: Install pytest and other packages needed for running tests
       (add your required selenium packages here)
    - name: Setup tmate session
    - name: Test build roundup and install locale so lang tests work.
    - name: Run tests

  • add one step to install chrome. You can look at the install postgres step in ci-test to see how to do it.
• remove the finish job as you won't be uploading coverage information.

[rouilj]

1) [ ] merge to main in test subdirectory of tracker 2) [ ] run selenium headless 3) [ ] identify/list requirements to run testsuite, packages/pip modules and document in README.md

[BharathKanama]

As I've pushed the testcases in Git, I'm closing this issue.