Re-visit end-to-end testing for Staging

Issue description

Incorporating end-to-end testing into our deployment process will provide us with a safeguard against new deployments breaking current application functionalities.

We conducted an initial investigation of end-to-end testing in Staging. In #1537, we looked at tools to support E2E testing and identified Crypress as a candidate for further exploration. In #2000, we talk to a team using Crypress for an understanding of it's pros and cons. The decision from the initial investigation was to hold off on implementation of E2E testing for the immediate future.

This ticket is created when the project deem it ready to revisit E2E testing. This ticket is to investigate how we can incorporate end-to-end testing for Staging.

Questions for considerations:

What tests should be included?
What types of testing can be doing? (e.g., manual, automated and hybrid)
Is automated browser-based tests using Cypress an option?
Libraries to consider (Selenium, testing-library)?

Acceptance criteria

[ ] review analysis from prior investigation
[ ] Assess feasibility of automated end-to-end testing
[ ] Present analysis, findings and final recommendations to team in engineering huddle or a separate meeting that the researcher sets up
[ ] Create dev tickets for implementation of POC
[ ] Create tickets for determining what we want to test

Additional context

more context on what another gsa team did for e2e snippet from eng huddle notes overview, see link to more indepth notes

End to End testing call with TTS FAC. Notes Deploy setup: Pushing to main will deploy to dev. Staging deploy every morning which includes regression tests with end to end testing. Prod manually deployed weekly.

General design advice

“E2E tests remain fragile. Once you start E2E testing, you need to have a culture on the team on creating it as a first class part of the app. It’s a substantial part of the product, a nontrivial amount of code, and if implemented poorly, it’s very fragile.”
Everyone on the team must know how E2E testing works because when the app changes, you need to make sure E2E testing has coverage.
Recommend page object model approach. It decently reflects how the user interacts with the app.
Note: Cypress itself has recommended app actions over page object model so we may need to see what best matches our app’s intended testing model.
Login.gov: FAC does not stub and instead generates a 2FA time-based one time password (TOTP). Uses a library that simulates a TOTP client. Testing Local testing: Run through docker-compose. People at various times have not been able to run tests locally because of machine incompatibility from not having containerization. These issues happen when you try to run local tests manually. Testing mostly done on staging. “Cypress does not containerize great as far as I can tell.”

Links to other issues
Related to: #1537
Related to: #2000

cisagov / manage.get.gov