[Discovery] Evaluate documentation for creating e2e tests on VSP

[accbjt]

https://department-of-veterans-affairs.github.io/veteran-facing-services-tools/getting-started/common-tasks/new-end-to-end-test

https://department-of-veterans-affairs.github.io/veteran-facing-services-tools/getting-started/common-tasks/debugging-tests/

https://department-of-veterans-affairs.github.io/veteran-facing-services-tools/getting-started/common-tasks/test

https://github.com/department-of-veterans-affairs/vets-website/blob/master/docs/Keyboard_e2e.md

https://github.com/department-of-veterans-affairs/vets-website/blob/master/README.md

https://github.com/department-of-veterans-affairs/vets.gov-team/blob/ee6aa46b6053ceb3cd437db4e8433feba123689b/Practice%20Areas/QA/Cross-Platform%20Testing/cross-platform-testing-plan.md

https://github.com/department-of-veterans-affairs/vets.gov-team/blob/ead0fe129bfee6b8e7eaae59d7f4f681a37dec15/Products/Disability/Disability%20526EZ/BAH-526/engineering/reporting/ancillary-forms-integration-testing.md#e2e-testing-overview

https://github.com/department-of-veterans-affairs/vets.gov-team/blob/ead0fe129bfee6b8e7eaae59d7f4f681a37dec15/Practice%20Areas/Accessibility/Readme.md

[gunsch]

Question on this: I see all the links you found, are we happy with that state of things? That seems like a lot of docs in a lot of places that don't really link to each other, and it's not clear to me which of those we'd point a VFS developer at when we say "here's how you should build e2e tests."

Trying to understand what next steps are here.

[U-DON]

I've documented the state of our E2E process and included the current recommendation.

Edit by @mnorthuis: new location for file above

I don't know where that doc should belong explicitly, and it isn't something we're pointing every VFS dev to, but in our documentation site for veteran-facing services tools, we do have a page on writing E2E tests. I think that's our most official recommendation for writing E2E tests.

For next steps, after experimenting with TestCafe, I believe we should proceed with it as our testing framework and with BrowserStack as our cross-browser cloud service. There are several parts to integrating them into our workflow and CI.

1. Figure out what BrowserStack account to use. I believe we have an organization account, but I don't know if it's on the free open source plan and whether it's for VA.gov specifically. When I tested, I created my own account, so I need to understand the situation of our existing account and how we manage ownership and admin roles.
   • Part of signing up for the open source plan involves an application, and another condition is that we need to display the BrowserStack logo in the repo in which we use it, so we should probably confirm whether that is okay by DSVA. But if the existing org account is specific to VA and is a paid plan that provides us much better features, I suppose we don't have to do that application.
2. Get the TestCafe tests to run in Jenkins on production deploys or production build PRs. As part of that, we should ensure the tests are running against our API (vets-api).
   • Need to loop in DevOps to understand how we can proceed with this. My impression is that we might be able to leverage a review instance generated in the production build PRs, but I don't have a full grasp on what we're able to do.
3. Set expectations on test coverage and how many of the current E2E tests we should convert to TestCafe. Perhaps set expectations on QA's role and the extent to which they may assist. A large fraction of the tests are forms, so we should also look into modifying the form filler utility (written in Puppeteer) to use TestCafe APIs.
4. Once all of the TestCafe process is integrated into our CI and well understood, we need to update the documentation so that the recommendation is to start writing tests in TestCafe. We should also remove the old tests and dependencies.

[U-DON]

I see that there were some changes made to this doc as part of this ticket, and I had no idea this documentation repo existed. I think there may be some duplication between the vets-website section of this repo and the front-end documentation site, and while I can see they each have their own purpose, I'm not quite sure how we should reconcile the duplicated content.

[smoonpeyes]

cc: @andreahewitt-odd

[accbjt]

@mnorthuis @gunsch Do you know where this document should live. Initially we wanted to keep all front end development documentation on the front end documentation site. We want one source of truth and we don't have a clear place for that yet. I don't know what the policy is for our GitHub and our documents needing to be in there.

[mnorthuis]

I'm not sure which of all the documents you are asking about, but agree with @gunsch, its a lot of links to a lot of different places. VA.gov-teams should be where we place all non-code documentation that helps the VFS teams learn how to work on the platform. That said, there is so much stuff in so many repos that I'm not sure that is entirely realistic for all things.

It would be really helpful if we could pull together a list of all the places that developer documentation is currently stored, and then start discussions around where it should all live.

Here is a list of repos I started, this might be a good place to start gathering information on what repos are used and what they generally contain. Then we can discuss what repos should continue to exist and what content should be archived or moved. We can throw this into another issue for another time as well.

https://docs.google.com/spreadsheets/d/1waFEAra9aeXK4UFp-ODMbCcKC4MHDZCFel8Oahb2X1E/edit?usp=sharing

[pjhill]

Tooling and culture has shifted since this ticket was created. This ticket has been OBE. E2E test documentation evaluation may still be relevant, but it should probably be pursued under a new ticket.