search

microsoft / TemplateStudio

Template Studio accelerates the creation of new WinUI 3, WPF, and UWP apps using a wizard-based experience.
Other
2.65k stars 459 forks source link

Documentation of build process and testing #4403

Open mrlacey opened 2 years ago

mrlacey commented 2 years ago

We need better documentation of the build process and pipelines, including where, when, and how different builds and tests are run.

The pipelines are in the repo but I can't see details of when and why they are run. Actually, I can see they are run on Azure [DevOps] Pipelines, based on https://github.com/microsoft/TemplateStudio#build-status but the documented description seems out of date.

For example, the main readme says that the CI Build "Runs every PR requested / PR accepted." Ignoring the bad grammar, this isn't true. There is a separate pipeline/configuration that is run for PRs. The PR-specific one runs many more tests than the CI one, which makes me wonder why the CI one exists and what value is expected from running it. Why not use the PR one instead?

For reference, I'm currently refactoring the tests as part of dividing the project into multiple extensions and am trying to balance how best to group them going forward while still learning from the past. My needing to know how they currently work has also highlighted the documentation that will also be needed going forward.

mikebattista commented 2 years ago

Thanks for splitting out the testing. That's definitely important and will make the build validation faster and more reliable too.

The triggers in the YAML show when they are run. The documentation is still applicable, the only changes I made so far were checking in YAML and pointing the pipelines at it. I'm sure the documentation could be improved and we should update it based on the new project structure. I think consolidating the dev/release branches into a single main branch will also help simplify the story and remove some of the legacy reasons for the different CI vs. PR pipelines.

PRs are currently gated on the PR pipeline passing, which is configured in the Branch Protection settings in the repo.

mikebattista commented 2 years ago

CI and PR pipelines were merged into just the CI pipeline with 5d8948cbd6e14ba71da123a5fbf2aea2f45fbf75 and cf3976f0fbb973b01468b3d9d1ec30e4f6db78ba.

Still need to refactor the various other test pipelines for the new project structure.

mikebattista commented 2 years ago

Build Status section was updated with https://github.com/microsoft/TemplateStudio/pull/4445 and describes the new CI and Nightly pipelines.

Is all that's left documenting Manual and Release tests? Where do you see that living?