Critical path flow descriptions for e2e testing

[sarayourfriend]

Due date: 2022-04-13

Assigned reviewers

• [ ] @obulat
• [ ] @krysal

Description

Snapshots should be taken at the "landing" (final-ish) parts of the flow. A leading ! indicates these steps in the flow for each path.

Homepage to related single result

This path implicitly includes search-to-result

Happiest path (multiple pages of results)

I like the "birds" results because the pictures are pleasant AND the audio results are not just dogs barking, but feel free to replace/supplement with the "dogs", "cats" etc searches we love as well. Variety can only help!

Definitions

These could probably be adopted as navigation helpers/utilities

• go-to-search(term: string, type?: 'image' | 'audio'):
  1. /
  2. /search?q={term}
  3. type && /search/{type}?q={term}

     All results

  4. go-to-search term: 'birds'
  5. ! /{image,audio}/:uuid
  6. ! /{image,audio}/:uuid (related result)

     Image - via in-result content type switcher

  7. go-to-search term: 'birds' type: 'image'
  8. ! /image/:uuid
  9. ! /image/:uuid (related result)

     Image - via header search type switcher

  10. go-to-search term: 'birds' type: 'image'
  11. ! /image/:uuid
  12. ! /image/:uuid (related result)

      Audio - via in-result content type switcher

  13. go-to-search term: 'birds' type: 'audio'
  14. ! /audio/:uuid
  15. ! /audio/:uuid

      Audio - via header search type switcher

  16. go-to-search term: 'birds' type: 'audio'
  17. ! /audio/:uuid
  18. ! /audio/:uuid

      Variants of the above

      For each of these, snapshot the last page of the flow, make sure to do "full page" screenshots.

  19. Single page of results (snapshot results page)
  20. No related results (is this possible?)
  21. Click load more on multiple pages of results and select a result from the new results (snapshot results after clicking load more)
  22. Same as 3 but select a result from the first page of results
  23. No results (snapshot results page)
  24. All of the above (including the happiest path) but get there using filters (snapshot the page with the filters open?)
  25. Variation of this is to apply a filter from each category for a search term we know will have
  26. All of the above but throw a refresh in the flow
  27. For example, refresh after the result is selected then hit the back button to the search page
  28. All of the above but also hit the back button from the related result all the way back to the search results page AND select another result
  29. All of the above but cut out earlier steps in the flow so that the "start" of the flow is in the middle of the top-to-bottom "full flow" (i.e., landing directly on a search results page or directly on a single result and navigating to a related result, etc).

[sarayourfriend]

Note that parts of the flow are already covered (we have tests from homepage to results page for example). We just want to augment the existing tests with end-to-end user flow tests.

[sarayourfriend]

@WordPress/openverse-maintainers please review this RFC/proposal :pray: It totally fell off my radar.

[obulat]

For each of these, snapshot the last page of the flow, make sure to do "full page" screenshots.

Would this mean that these tests are visual-regression tests, not e2e tests in our terminology, @sarayourfriend?

[sarayourfriend]

I guess they'd be doing both. I'm not totally sure if it makes sense to separate them for these "long flow" type tests because we'd want to confirm they work in the contexts of a user actually going through the core flows.

On the other hand, we would need to have some way to disable them when testing against staging otherwise the visual regression tests will end up needing to somehow ignore differences between the tape data and production data. That's an issue we don't need to solve now though.

Do you think there's a good reason to separate these particular snapshots out into their own tests (that still preserves their ability to test the visual results of full user flows)? Right now our visual regression tests run at the same time as the other e2e tests. I'm not totally sure the distinction is that important. What do you think?

[sarayourfriend]

@obulat Do you think this is still worthwhile to pursue? If so, it might help to break it into smaller issues with help wanted labels?