CI/CD of Satellite UI: Automated UI Tests

ferristocrat commented 1 year ago

Description:

The goal is to improve the engineering team's workflow by integrating automated UI tests into the CI/CD pipeline of the Satellite UI. By automating UI testing, we can reduce manual testing efforts, increase confidence in the stability and quality of the UI, and speed up the delivery of new features and updates.

Problem/Pain Point:

Manual UI testing is time-consuming and can be error-prone, leading to missed issues and potential regressions in the Satellite UI. Furthermore, relying on manual testing can slow down the development and release process, impacting overall productivity and efficiency within the engineering team. The limited size of our QA team compared to the engineering team makes this problem even more pronounced.

Why Now

As the Satellite UI continues to evolve and new features are added, the importance of a reliable and efficient testing process becomes more critical. Implementing automated UI tests now will help ensure the stability and quality of the Satellite UI, enabling the engineering team to deliver new features and updates more rapidly and with greater confidence, while also alleviating the burden on the QA team.

Impact

Integrating automated UI tests into the CI/CD pipeline will increase the overall quality and stability of the Satellite UI by quickly identifying and addressing issues. It will also improve the engineering team's efficiency by reducing the time spent on manual testing, accelerating the release process, and reducing the workload on the smaller QA team.

In Scope:

Determine key functionality that we want to include as the minimum tests
Development and implementation of automated UI tests for key functionality and user flows within the Satellite UI.
Integration of the automated UI tests into the existing CI/CD pipeline.
Documentation of the automated UI testing process and best practices for the engineering team.

Out of Scope:

Automated deployments (whether staging or production)

Success Metrics

Reduction in the number of UI-related issues and bugs reported by customers and internal users.
Decrease in the time required for the QA team to test and validate UI changes.
Increase in the efficiency of the CI/CD pipeline, as measured by shorter build and deployment times.
Positive feedback from the engineering and QA teams on the new automated UI testing process.

Links:

Milestone(s): https://github.com/storj/storj/milestone/34

ROM: 3 sprints - 1 sprint getting into Storj repo, 1 sprint getting into CI/CD, 1 sprint for ensuring we have correct tests in place

iglesiasbrandon commented 1 year ago

@mobyvb @NiaStorj @ferristocrat can you all update this description during your next grooming call?

Please update the title to include or not include "(and maybe automatic staging environments)".

Remove any of the ### headers in the ticket description that you don't need (they are just boiler plate)

ferristocrat commented 8 months ago

This is not currently being worked on, and the Vuetify work may change the scope of this item. Moving to "Idea" for now.

iglesiasbrandon commented 4 months ago

Hey @ferristocrat @mobyvb is this roadmap item still relevant with the transition to satellite UI v2? it seems like we should have automated tests for all new features as a part of the development process.

mobyvb commented 4 months ago

@iglesiasbrandon yes, this roadmap item is still relevant. We actually recently got the UI tests to execute for Gerrit changes as part of this task: https://github.com/storj/storj/issues/5695 However, it seems that there are still situations where the UI tests are flaky (we specifically noticed this on Github changes). Still debugging that, but things are mostly working. Eventually we will want to increase the scope of the UI tests - right now they only test some basic signup/login/onboarding/upload/download features, but there are other things, like access grants, team invitations, etc... that are untested.

Anyway, we may want to revisit this roadmap item now that we have made a lot of progress with getting the UI tests to execute. We can probably get more specific about the AC.

storj / roadmap