Windows tests should not checkout `main` branch and merge the PR branch

rm3l commented 2 years ago

/kind bug /area testing /area Windows

What versions of software are you using?

Output of odo version:

odo v3.0.0-rc2 (f78096d26)

Server: https://10.10.10.105:6443
Kubernetes: v1.24.6

Actual behavior

Currently, Windows tests checkout the main branch and then merge the PR branch into the local main branch: https://github.com/redhat-developer/odo/blob/f78096d26430e599641022cddef51eb4481b52eb/.ibm/pipelines/windows-test-script.ps1#L31-L34

This can lead to potential issues for several reasons:

it does not guarantee that the code tested is the one from the PR branch. At the time the test is run, if the PR branch is behind main, with main containing changes not conflicting with the PR branch, the merge operation would succeed. But this would give a local branch with changes coming from both main and the PR branch. I think this explains the behavior I observed and reported in https://github.com/redhat-developer/odo/pull/6108#issuecomment-1249215452 (where a given test spec had run, but the PR branch had no reference to that test spec) and which led me to file #6131 (for better traceability).
At the time the test is run, if there are conflicts between main and the PR branch, the whole Windows test would not pass.
This might prevent from testing PRs targeting branches other than main, like release/v3.0.0.

To me, the test should run with what is exactly in the PR branch, and not attempt to merge anything (or if this intended, it should be a dedicated stage in the Pipeline). We already have the OpenShift CI bot reporting potential conflicts and requesting to rebase PRs.

Expected behavior

Just like what is done in Kubernetes and OpenShift tests, Windows tests should rely on the code already checked out by IBM Cloud. We could scp the current folder to the Windows machine, as we do with the PowerShell script: https://github.com/redhat-developer/odo/blob/main/.ibm/pipelines/windows-test.sh#L13. Or if the scp operation is too slow, the Windows test script could perform a shallow checkout of the right PR branch (rather than merging it into the local main branch).

rm3l commented 2 years ago

Fixed by #6177 (and merged into release/v3.0.0). This will get merged into main right after the v3.0.0 tag.

/close

openshift-ci[bot] commented 2 years ago

@rm3l: Closing this issue.

In response to [this](https://github.com/redhat-developer/odo/issues/6175#issuecomment-1265123984): >Fixed by #6177, which will get merged into `main` right after the `v3.0.0` tag. > >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.

redhat-developer / odo