canonical / test_observer

Flutter based dashboard for visualising SRU regression test results
3 stars 3 forks source link

Add rerun test executions helper script #195

Closed omar-selo closed 2 months ago

omar-selo commented 4 months ago

Description

This is a script helper script for reviewers. Often when reviewing an artefact, we'd want to rerun all test executions that failed a particular test or group of tests. This handy script does just that for us as otherwise we'd have to manually go through all of the test executions in the UI which is time consuming and annoying.

$ python scripts/rerun_failing_test_executions.py -h
usage: rerun_failing_test_executions.py [-h] artefact_id test_case_regex

Reruns test executions of an artefact that failed particular test cases matched by the passed regex

positional arguments:
  artefact_id
  test_case_regex

options:
  -h, --help       show this help message and exit

Example:

  python rerun_failing_test_executions.py 47906 '.*wireless.*'
  python rerun_failing_test_executions.py 47906 '.*suspend.*'

Resolved issues

Resolves https://warthogs.atlassian.net/browse/RTW-337

Documentation

None.

Web service API changes

None.

Tests

This is a throwaway script (ideally this sort of functionality will be available through the UI eventually). But I've tried it myself a few times.

omar-selo commented 4 months ago

Example output:

$ python scripts/rerun_failing_test_executions.py 47225 '.*wireless.*'
will rerun dell-latitude-7350-c32372 for failing after-suspend-wireless/wireless_connection_wpa_bg_nm_wlp86s0f0
will rerun dell-unidentified-system-c32236 for failing wireless/wireless_connection_open_n_nm_wlp0s20f3
will rerun hp-eliteone-870-27-inch-g9-all-in-one-desktop-pc-c29775 for failing wireless/wireless_connection_wpa_n_nm_wlp0s20f3
will rerun hp-proone-440-238-inch-g9-all-in-one-desktop-pc-c29752 for failing after-suspend-wireless/wireless_connection_open_n_nm_wlp0s20f3
will rerun hp-proone-440-238-inch-g9-all-in-one-pc-c31581 for failing wireless/wireless_connection_open_n_nm_wlp7s0
will rerun hp-proone-440-238-inch-g9-all-in-one-pc-c31582 for failing wireless/wireless_connection_wpa_n_nm_wlp0s20f3
will rerun hp-proone-440-238-inch-g9-all-in-one-pc-c31584 for failing wireless/wireless_connection_open_n_nm_wlp0s20f3
Will rerun 7 test executions is that ok? (Y/n) Y
Submitted rerun requests