Migration of testing to Cypress in a duplicate repository.
💬 Description
This issue completes the work for the initial technical setup for the move to Cypress for all forms of testing, this includes:
unit (decided not in the scope of this issue)
e2e
visual regression
performance
accessibility
React
This should include tests for both Web Components and React. However, some thought should be given to the scalability of having to duplicate some tests in React, if that is even useful. For example, for Slotted SVG, it would be useful as otherwise it would not be covered. Consideration here for the effort required to duplicate the tests initially, but also the time it would take to run all the different types of tests in the pipeline.
This issue should not only include the setup of all kinds of tests, but also the following:
duplicating the ic-ui-kit repo to set this up, allowing us to not affect production as we test
documenting the setup required for when we move over the setup to the production ic-ui-kit in a later issue
examples of each type of test in 2-3 components of differing complexity (Button, Top Navigation and Select), ideally covering most of the test cases the current tests do, especially the ones that may cause issues
GitHub Actions setup to run all forms of tests listed above in headless mode
is it useful to host the Cypress report as part of the GitHub Actions or are other options suffice?
can we add a way to test for High Contrast Mode and other features similar?
are there any potential blockers we can envisage for when we move across the entire library? (e.g. running visual tests on all browsers could be incredibly slow for the GitHub Actions to pass)
confirm at the end, that tests run on both Linux and Windows
As a final output of this issue, the following two issues need to be created:
Move over this work into the production ic-ui-kit (including the documented steps taking as part of this issue)
Issues for each component to move all tests over to Cypress
Removal of all traces of the old testing frameworks from dependencies and the code, including updating any documentation or READMEs where relevant
💰 Use value
To initially test if Cypress works for our use case (StencilJS/web components), and if there are any blockers to our adoption, before we move this work into production.
Long term, this will allow us to simplify our testing stack to do all our tests with one technology (Cypress) as opposed to using various (Jest, Puppeteer, Loki, Axe etc), but add the additional benefits of performance testing and fixing issues with our current visual testing.
Overall, this will improve our product quality.
Additional info
This issue sets the ground work for the follow up issue of moving it across to the ui-kit, so it'd be best to set it up in a way where it can be copied across. After that, there will be more issues to work through all components to move tests across.
Summary
Migration of testing to Cypress in a duplicate repository.
💬 Description
This issue completes the work for the initial technical setup for the move to Cypress for all forms of testing, this includes:
unit (decided not in the scope of this issue)This should include tests for both Web Components and React. However, some thought should be given to the scalability of having to duplicate some tests in React, if that is even useful. For example, for Slotted SVG, it would be useful as otherwise it would not be covered. Consideration here for the effort required to duplicate the tests initially, but also the time it would take to run all the different types of tests in the pipeline.
This issue should not only include the setup of all kinds of tests, but also the following:
As a final output of this issue, the following two issues need to be created:
💰 Use value
To initially test if Cypress works for our use case (StencilJS/web components), and if there are any blockers to our adoption, before we move this work into production.
Long term, this will allow us to simplify our testing stack to do all our tests with one technology (Cypress) as opposed to using various (Jest, Puppeteer, Loki, Axe etc), but add the additional benefits of performance testing and fixing issues with our current visual testing.
Overall, this will improve our product quality.
Additional info
This issue sets the ground work for the follow up issue of moving it across to the ui-kit, so it'd be best to set it up in a way where it can be copied across. After that, there will be more issues to work through all components to move tests across.