Add our tests to CI pipeline 🧪

camelPhonso commented 5 months ago

Expected Behavior

With Storybook now in our stack with component tests via the jest extension, plus the End-to-end testing via Cypress suites, we should run these in our CI pipeline. The simpler set up would be to run all tests for each PR merging into dev

Current Behavior

Tests are only run if developers trigger them manually.

Any other notes

No response

Assignment

This issue is free for anyone to take

mnixo commented 4 months ago

!request

github-actions[bot] commented 4 months ago

🤖 meep morp!

Thank you for your contribution @mnixo - this Issue is assigned to you as requested ✨

🔎 Please keep in mind that in order to give everyone a chance to contribute Issues may be unassigned if they go stale. You will be warned if the Issue is marked as stale at any point.

🔎 Please review our documentation for guidance on:

common Patterns we follow in the code base
conventions set for commit messages and naming of functions, classes or files

💡 The core team is available for any help and will try to respond to any comments in this Issue or questions on discord as quickly as possible.

Happy coding ! 🦆

mnixo commented 4 months ago

Storybook facilitates the integration of different types of tests, some of them including:

Visual tests.
- Based on Chromatic.
- Focuses on visual regression testing, with the goal to catch unintended visual changes that could go unnoticed in other test types.
- The Chromatic dashboard is feature-rich but could be cumbersome in a collaborative context.
- Probably not as important as other types of tests.
- Intended changes will still have to be reviewed manually.
Interaction tests.
- Based on play functions. More play functions = more component documentation + more coverage.
- Very functional.
- Easy to follow in the Storybook UI.
- The project already has a few play functions.
User flow tests.
- Based on Cypress.
- Nice DX with the Cypress UI.
- Solid E2E option integrated in Storybook.
- Don't need Supabase to run.
- Would be nice to check how "E2E" they really are with a real use case.
- Although there are already Cypress tests in the project, they might not be appropriate in this context.

It is important to note that these tests run in the context of Storybook, so there might be a few caveats.

For future reference, this PR has some exploratory work on how to integrate these in the project's CI.

camelPhonso commented 4 months ago

Closed this one as the PR associated has been merged.

But I'm curious on your take from looking at the user flow tests. To me it looks like having interaction tests for all our components and cypress e2e for the different user stories associated with each Milestone would be the better way to get decent coverage and the user flow stuff seems a bit redundant - specially if it actually ignores the context of the app 🤔

mnixo commented 4 months ago

To me it looks like having interaction tests for all our components and cypress e2e for the different user stories associated with each Milestone would be the better way to get decent coverage and the user flow stuff seems a bit redundant - specially if it actually ignores the context of the app 🤔

There's definitely a considerable overlap between the two. Having a "real" E2E context (with Supabase) would definitely be better. In this case it seems that the user flow tests are just "as E2E as you can get" just within the Storybook context.

enBloc-org / kindly