Spike: Understand level of effort to seperate test runs

[gracekretschmer-metrostar]

User Story or Problem Statement

As the PO of CMS, I need to understand the level of effort and skills needed to separate test runs to determine if/how to prioritize the work for the CMS team.

Description or Additional Context

One of the opportunities for the comprehensive test plan is to separate out the test runs, to ensure that developers are getting fast feedback on their code when tests are run. Before we move forward with that work, we need to scope out the skillset needed and level of effort to do this work.

Relevant Links

• https://github.com/department-of-veterans-affairs/va.gov-cms/issues/19004

Steps for Implementation

• Identify and outline options for segmenting the existing cypress test suite into separate test runs
• Discuss with devops and drupal engineers which method of segmenting the test suite best suits our needs
• Decide on an method, identify required skillsets for implementation (QA, devops, etc.), detail level of effort, and detail steps for implementation

Acceptance Criteria

• [ ] A plan of action, include level of effort and skillset needed for CMS team to separate the test runs is represented in Github.

[keisterj-oddball]

The current plan is to implement a critical path test suite using tags (similar to those used for skipping tests currently). @edmund-dunn, @timcosgrove, @7hunderbird and I have decided it makes the most sense to continue running the full cypress test suite against individual PRs in tugboat, and running the critical path suite in the Post-Deploy Tests job after staging deploys.

To implement, tags will need to be added to individual test scripts if they are considered critical path, an environment specific config will need to updated in the devops repo, and a script will need to be updated within the cms repo to filter out feature files that contain no critical path tests. My assessment is that the largest effort here is analyzing all existing tests and determining if they should be deemed critical path. This will likely be a joint effort between QA and drupal/devops engineers, and may require input from sitewide team members.