As we are doing a lot of integration testing, the objective is to have less e2e tests as a result. For the e2e tests we do have I think they should be covering key user flows in the application. While our app is small, there are flows we could define and then automate, checking multiple layers of the application rather than mocking.
Doesn't cover the ETL scripts, only the UI, API and database
Automated using Playwright
user end to end flows are documented in the wiki (use Gherkin style)
Use Cucumber BDD to enable the test to be written in business facing language
As there has been a lot of test effort targeted lower down the pyramid at integration points, the E2E tests can be very simple, just check that the system is available and that it works all joined together
Need to manually verify that the prod data in the summary table is correct for a given city
discovered a possible discrepancy of up to 3.5% between the prod summary table and the database data (via manual calculation) for London (Time Range: 18 Aug 12:00 - 19 Aug 12:00 UTC)
Need to manually check a specific graph is correct for the prod data
In the E2E tests we can then rely on using hard coded prod data / snapshots to verify that the app is working correctly
given the discrepancy in the summary table data, I will proceed with this approach unless the investigation into the possible bug reveals it to be an issue
Scenarios:
Default behaviour: navigate to homepage, filter to a city, check summary table values, click on city, check graph snapshot removing as we will need to use hardcoded data
Change forecast base time: navigate to homepage, change base time, filter to a city, check summary table values, click on city, check graph snapshot
Change forecast base time: navigate to homepage, change base time & forecast window, filter to a city, check summary table values, click on city, check graph snapshot
Change forecast base time: navigate to homepage, change base time & forecast window, filter to a city, check summary table values match with manually verified values from the database
Not enough time available to flesh this ticket out further, implementation could be potentially refactored
As we are doing a lot of integration testing, the objective is to have less e2e tests as a result. For the e2e tests we do have I think they should be covering key user flows in the application. While our app is small, there are flows we could define and then automate, checking multiple layers of the application rather than mocking.
Doesn't cover the ETL scripts, only the UI, API and database Automated using Playwright user end to end flows are documented in the wiki (use Gherkin style)
Test Checklist: