Closed gracekretschmer-metrostar closed 2 months ago
PRIORITY 1: Increase critical path test coverage
Pros:
Increasing critical path test coverage will help ensure the most important functionality in the CMS works as intended after each new change is introduced.
Reduces the need for manual testing in cases requiring full regression testing efforts.
Considerations:
Adding 41 tests to the "unsegmented" test suite may significantly impact the duration of test runs (impacts developer speed and build pipelines).
Requires prioritization of content types to be automated.
Likely requires Sitewide involvement for test creation.
PRIORITY 2: Separate existing test suite into "critical path" and "regression" test runs
Pros:
Prioritizing the "critical path" test run could speed up the build pipeline.
Regression could still be run on a regular schedule outside of the CMS pipeline providing frequent feedback on new changes.
Developers could optionally run full regression or critical path in PR environments based on the risk of changes being introduced.
Considerations:
This would require some input from Sitewide about where existing tests belong.
Running only "critical path" tests in CI without increasing critical path coverage reduces the amount of test coverage for new changes in the short term, although the value provided by some of the existing tests could be debated.
Could likely be completed internally by the CMS team.
PRIORITY 3: Persist test results outside of GH and Jenkins
Pros:
This will assist in the assessment of test result trends and identification of flaky tests.
This will enable the CMS team to identify common types of test failures and develop run books on how to address them.
Considerations:
I will create and link a separate ticket for each priority listed above before closing this ticket.
Note to self: remove from sprint 18.
User Story or Problem Statement
The CMS team needs to determine the priorities for filling in the testing gaps for critical pathways within CMS and how to implement the comprehensive test plan to move from current state of testing to ideal state of testing.
Description or Additional Context
The following opportunities have been identified by @keisterj-oddball:
Reference Links
Steps for Implementation
Acceptance Criteria