search

openshift / origin

Conformance test suite for OpenShift
http://www.openshift.org
Apache License 2.0
8.44k stars 4.69k forks source link

NO-JIRA: add conditional execution of featuregated tests #28670

Closed deads2k closed 1 month ago

deads2k commented 1 month ago

By adding [OCPFeatureGate:MyFeatureGate] to a test description, openshift-tests will filter the list using the list contained in FeatureGate. This allows writing the test once and it will run when enabled using any FeatureSet or ClusterProfile.

As a side-effect, it now becomes possible to count how many tests we have for a give feature.

/assign @dgoodwin

deads2k commented 1 month ago

/hold

holding. We should see this passing in TechPreview jobs and skipped in Default jobs.

deads2k commented 1 month ago

/help /test

openshift-ci[bot] commented 1 month ago

@deads2k: The /test command needs one or more targets. The following commands are available to trigger required jobs:

The following commands are available to trigger optional jobs:

Use /test all to run the following jobs that were automatically triggered:

In response to [this](https://github.com/openshift/origin/pull/28670#issuecomment-2023494096): >/help >/test Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
deads2k commented 1 month ago

/test e2e-gcp-ovn-techpreview

openshift-ci-robot commented 1 month ago

@deads2k: This pull request explicitly references no jira issue.

In response to [this](https://github.com/openshift/origin/pull/28670): >By adding `[OCPFeatureGate:MyFeatureGate]` to a test description, openshift-tests will filter the list using the list contained in FeatureGate. This allows writing the test once and it will run when enabled using any FeatureSet or ClusterProfile. > >As a side-effect, it now becomes possible to count how many tests we have for a give feature. > >/assign @dgoodwin Instructions for interacting with me using PR comments are available [here](https://prow.ci.openshift.org/command-help?repo=openshift%2Forigin). If you have questions or suggestions related to my behavior, please file an issue against the [openshift-eng/jira-lifecycle-plugin](https://github.com/openshift-eng/jira-lifecycle-plugin/issues/new) repository.
deads2k commented 1 month ago

/test e2e-gcp-ovn-techpreview

deads2k commented 1 month ago

looks to work correctly. Devan was ok in concept.

deads2k commented 1 month ago

/hold cancel

openshift-ci[bot] commented 1 month ago

[APPROVALNOTIFIER] This PR is APPROVED

Approval requirements bypassed by manually added approval.

This pull-request has been approved by: deads2k

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - **[OWNERS](https://github.com/openshift/origin/blob/master/OWNERS)** Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment
deads2k commented 1 month ago

z-networking][invariant] alert/OVNKubernetesResourceRetryFailure should not be at or above info again

/override ci/prow/e2e-aws-ovn-serial

openshift-ci[bot] commented 1 month ago

@deads2k: Overrode contexts on behalf of deads2k: ci/prow/e2e-aws-ovn-serial

In response to [this](https://github.com/openshift/origin/pull/28670#issuecomment-2024028783): >z-networking][invariant] alert/OVNKubernetesResourceRetryFailure should not be at or above info again > >/override ci/prow/e2e-aws-ovn-serial Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
deads2k commented 1 month ago

/override ci/prow/e2e-gcp-ovn-upgrade

failed on unrelated

openshift-ci[bot] commented 1 month ago

@deads2k: Overrode contexts on behalf of deads2k: ci/prow/e2e-gcp-ovn-upgrade

In response to [this](https://github.com/openshift/origin/pull/28670#issuecomment-2024037816): >/override ci/prow/e2e-gcp-ovn-upgrade > >failed on unrelated Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
openshift-trt-bot commented 1 month ago

Job Failure Risk Analysis for sha: 8207a5406e7138c75f35425e3646c8cdf3e92c8a

Job Name Failure Risk
pull-ci-openshift-origin-master-e2e-aws-ovn-single-node-upgrade IncompleteTests
Tests for this run (100) are below the historical average (1664): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-aws-ovn-single-node-serial IncompleteTests
Tests for this run (99) are below the historical average (559): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-aws-ovn-single-node IncompleteTests
Tests for this run (99) are below the historical average (1267): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-aws-ovn-serial Medium
[bz-networking][invariant] alert/OVNKubernetesResourceRetryFailure should not be at or above info
This test has passed 95.83% of 24 runs on jobs ['periodic-ci-openshift-release-master-nightly-4.16-e2e-aws-ovn-serial' 'periodic-ci-openshift-release-master-ci-4.16-e2e-aws-ovn-serial'] in the last 14 days.

Open Bugs
Help needed understanding 4.16 upgrade duration increase on metal
openshift-ci-robot commented 1 month ago

/retest-required

Remaining retests: 0 against base HEAD 3515349dbb6558a379d125b11f0d25b22e1ab641 and 2 for PR HEAD 8207a5406e7138c75f35425e3646c8cdf3e92c8a in total

deads2k commented 1 month ago

/override ci/prow/e2e-gcp-ovn-builds

openshift-ci[bot] commented 1 month ago

@deads2k: Overrode contexts on behalf of deads2k: ci/prow/e2e-gcp-ovn-builds

In response to [this](https://github.com/openshift/origin/pull/28670#issuecomment-2024245118): >/override ci/prow/e2e-gcp-ovn-builds Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
openshift-ci[bot] commented 1 month ago

@deads2k: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-aws-ovn-single-node-upgrade 8207a5406e7138c75f35425e3646c8cdf3e92c8a link false /test e2e-aws-ovn-single-node-upgrade
ci/prow/e2e-metal-ipi-sdn 8207a5406e7138c75f35425e3646c8cdf3e92c8a link false /test e2e-metal-ipi-sdn
ci/prow/e2e-aws-ovn-single-node 8207a5406e7138c75f35425e3646c8cdf3e92c8a link false /test e2e-aws-ovn-single-node
ci/prow/e2e-agnostic-ovn-cmd 8207a5406e7138c75f35425e3646c8cdf3e92c8a link false /test e2e-agnostic-ovn-cmd
ci/prow/e2e-aws-ovn-single-node-serial 8207a5406e7138c75f35425e3646c8cdf3e92c8a link false /test e2e-aws-ovn-single-node-serial

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).