search

nasa / openmct

A web based mission control framework.
https://nasa.github.io/openmct/
Other
12.06k stars 1.25k forks source link

[e2e] Speed up e2e test execution to get us back under the 5 min mark #6586

Open unlikelyzero opened 1 year ago

unlikelyzero commented 1 year ago

Summary

At present, our e2e tests can take at least 9 minutes to finish for the following reasons:

  1. Limited to 4 parallel workers
  2. Increased number of tests marked a stable (good thing)
  3. Test retries and flaky tests
  4. Parallelization of test suites now use # of files instead of # of testcases

We've recently implemented DeploySentinel to keep track of our retries and flaky tests to better track #3. Attempting to increase the number of workers didn't have the intended affect because of #4, above.

unlikelyzero commented 1 year ago

After some investigation, it turns out that the only path forward is to start splitting large test files: playwright/test parallelizes by test file and not testsuite/testcases