dashboard: Indicate if a test was rerun

[sprt]

Context: Sometimes a nightly/PR workflow is rerun manually (because some test may be flaky and a retry may succeed). In the dashboard we want to display this info, and not just show the latest run.

Challenges:

• The API might not let us fetch that data, TBD.

Acceptance criteria:

• [ ] Ideally, in the UI instead of showing one ✅ or ❌, we show as many icons as there were runs of that specific test
  • For example, 2 failures and 1 success = ❌❌✅
  • Note: it could be that the entire testsuite is rerun, or that only one or a few specific tests are rerun, make sure you handle all cases

Known limitations:

• We're still only fetching the data every 24 hours, so the data may still be stale. This is OK, as long as we're capturing ALL previous runs whenever data fetching happens.

[afinn12]

Code in: PR #22

Description

• Rerun column added to table that counts the total number of times the job/check was reran

• Tooltip menu that displays the results for each rerun on hover

• Split fetch scripts for nightly/PR data

  • Added yaml for PR fetch script
  • Added triggers to both yamls on pushes to main (data pulled again if new data is needed in changes to files)

• Also added authentication

  • To to use, put token in .env file
  • TODO: Make secret so fetch files still work in prod

• TODO: Include rerun stats in column stats? (i.e. fails, skips)

  • TODO: Even if all tests fail/pass, not added to fails/skips (suggests a pass)

• TODO: Also, given the nightly view picture, note that one job was rerun 5 times and one was rerun 4 times. Should the rerun count be 9 (5+4), or 7 (4+3), since runs = 9? Right now, sets reruns as 9 in this scenario.

• TODO: Only get PRs with ok-to-test label (so keep pulling closed PRs until get 10 with that label)

Nightly View

PR View