search

stolostron / grc-ui

Apache License 2.0
0 stars 16 forks source link

Enable fail-fast mode for BVT #869

Closed JustinKuli closed 2 years ago

JustinKuli commented 2 years ago

Signed-off-by: Justin Kulikauskas jkulikau@redhat.com

openshift-ci[bot] commented 2 years ago

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: JustinKuli

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/stolostron/grc-ui/blob/main/OWNERS)~~ [JustinKuli] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment
ycao56 commented 2 years ago

Fail fast was intentionally off for canary. Now since canary is moving to BVT, I assume this is asked by CICD team?. If yes, we can just update https://github.com/stolostron/grc-ui/blob/52943355e051f508e2f3fffd0f97aae3a2f0fce0/build/run-docker-tests.sh#L45-L50 Today, it is fail fast everywhere except canary.

sonarcloud[bot] commented 2 years ago

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

ycao56 commented 2 years ago

@JustinKuli is this for 2.4 or main? The test won't work when grc-ui pod is pulled out of the build. So the test needs to be removed from canary

openshift-ci[bot] commented 2 years ago

@JustinKuli: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-tests b1a52b175865bc8ba83b4776fc03359ed3cedd61 link true /test e2e-tests

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
JustinKuli commented 2 years ago

@ycao56 this is for current BVT. I hadn't noticed that other configuration point, but that would make more sense. We would want fail-fast on BVT. SVT would still run all of them.

The background is that when the policy framework is very unhealthy (recently happening more often because of changes in the way things are deployed), the UI tests will take 3+ hours and timeout the job. Because of the way it times out, it messes up test reporting for the entire product.

JustinKuli commented 2 years ago

@ycao56 is this a better approach? https://github.com/stolostron/canary-scripts/pull/252