How to better QA UI release? - so we can avoid cutting several patches while releasing the instance

[hanbyul-here]

Context

Even though VEDA UI holds most of the core features of the instance, the VEDA-UI release is not getting proper QA before the release. As a result, some of the bugs were found only when GHG instance was trying to use the release. We had to cut several hot-fix patch releases as a result of it: https://github.com/NASA-IMPACT/veda-ui/releases (5.1.2, 5.1.1, 5.0.1 were hotfixes ).

Assuming we cannot get as many resources for UI release QA as for GHG release, how can we improve UI QA? Here are some ideas.

Ideas

Please feel free to edit, or add comments.

Dev side

• Make datasets we use for dev as close as possible to the real datasets: We haven't updated our mocked datasets for a while. We can copy the dataset configuration files from the instances.
• Trigger the preview from the instance side from UI release: I am not sure how feasible it is, but it might be helpful to make an automatic PR with the latest UI version from the instance side so that we can test the UI version from the instance immediately.

Product side

• Make several scenarios that we should test. We can start with a simpler scenario like @j08lue listed out in this comment: https://github.com/NASA-IMPACT/veda-config/pull/411#pullrequestreview-2136547932 and add more cases that have required more of our attention before. (A side note that this can be a valuable resource for more robust e2e testing @dzole0311 suggested)

Acceptance Criteria

• [ ] Given when there are action items for better QA for UI release

Related Tickets

[If applicable, link any tickets that are related]

[j08lue]

Thanks for calling this, @hanbyul-here, I thought the same after the latest release.

Suggestion: Release candidate testing

I think a simple process change could already improve things:

1. Develop changes to VEDA UI
2. Publish changes to VEDA UI under a pre-release tag (e.g. using a release candidate version like 5.2.0-rc.1)
3. Create instance previews (primarily GHG Center) and share with instance-level testers
4. When instance preview testing was successful:
   1. Create a VEDA UI release
   2. Update the instance release candidate preview to the release preview
   3. Merge to staging on the instance
   4. Test staging on the instance to be extra sure

Rationale

Some issues only show up with real content and the instance-level testers have the sharpest eyes to spot them, since they are familiar with the content. We should make use of that testing capacity we have.

I was astonished that the GHG Center upgrade of VEDA UI for this release was merged to staging before extensive review. Was that how we expect things to go?

• https://github.com/US-GHG-Center/veda-config-ghg/pull/411

I think we should test before we merge to staging. Having bugs on staging is dangerous and an obstacle to by-passing blocking bugs, if necessary.