qentinelqi / qweb

Keyword driven automation for the web
https://github.com/qentinelqi/qweb
Apache License 2.0
39 stars 17 forks source link

Parallelize acceptance tests #147

Closed turunenm closed 4 months ago

turunenm commented 5 months ago

Parallellize acceptance tests with pabot for local execution (duty) and gh actions. Uses default of process per core which might require tuning for gh actions. Local exec time with 8 processes is around 5,5 minutes.

Test suites have been divided to serial and parallel dirs. Any test that requires a screen should be placed into the serial dir and headless tests in to the parallel dir. If a test suite had both headless tests and tests requiring a window, the suite has been split to [test]_headless and [test]_windowed.

A utility script _test/acceptance/pabot_suiteordering.py for creating pabot suite ordering on the fly is ran both in gh action and in duty to group the serial and and parallel tests.

Added use of http server for local .html files to get MacOS tests to pass. This meant changing all relative .html filepaths to urls @127.0.0.1:8000 In gh action this is done with an action called http-server-action and in the acceptance_tests duty python http.server is started as a subprocess which is terminated after running the tests.