Closed openshift-bot closed 3 days ago
@openshift-bot: This pull request explicitly references no jira issue.
alerts Information
There were (
1926
) added jobs and (30
) were removed.Comparisons were above allowed leeway of
30.00%
Note: For P99, alerts had
47
jobs increased and299
jobs decreased.Click To Show Table
| Name | Release | From | Arch | Network | Platform | Topology | Job Results | P95 | P95 % Increase | P99 | P99 % Increase | | ---- | ------- | ---- | ---- | ------- | -------- |--------- | ----------- | --- | -------------- | --- | -------------- | | TechPreviewNoUpgrade openshift-kube-apiserver-operator Warning | 4.18 | | amd64 | ovn | metal | ha | 298 | 1h16m5.8s| 32.92% | 1h20m33.02s | 30.64% | NodeSystemSaturation openshift-monitoring Warning | 4.18 | | amd64 | ovn | aws | single | 122 | 6m7.5s| 101.92% | 8m52.74s | 93.98% | HighOverallControlPlaneCPU openshift-kube-apiserver Warning | 4.18 | | amd64 | ovn | gcp | ha | 143 | 2m53s| 198.28% | 4m0.8s | 83.12% | NodeSystemSaturation openshift-monitoring Warning | 4.18 | 4.18 | amd64 | ovn | aws | single | 185 | 2m45.6s| 88.18% | 3m34.96s | 45.24% | HighOverallControlPlaneCPU openshift-kube-apiserver Warning | 4.18 | 4.18 | amd64 | ovn | gcp | ha | 274 | 1m28s| 214.29% | 1m58s | 103.45% | CannotRetrieveUpdates openshift-cluster-version Warning | 4.17 | | amd64 | ovn | aws | ha | 128 | 0s| 0.00% | 14m47.68s | 1205.41% | KubeJobFailed openshift-insights Warning | 4.17 | | amd64 | ovn | aws | ha | 128 | 59m29.9s| 0.00% | 2h51m29.86s | 66.73% | KubeAPIErrorBudgetBurn openshift-kube-apiserver Critical | 4.17 | | amd64 | ovn | aws | ha | 128 | 0s| 0.00% | 1m41.8s | 2093.97% | TargetDown openshift-ingress Warning | 4.17 | | amd64 | ovn | aws | ha | 128 | 0s| 0.00% | 31m8.8s | 818.96% | KubeletPlegDurationHigh kube-system Warning | 4.18 | | amd64 | ovn | gcp | ha | 143 | 0s| 0.00% | 58s | 360.32% | KubeJobFailed openshift-insights Warning | 4.18 | | amd64 | ovn | aws | ha | 321 | 1h43m6s| 0.00% | 3h12m16.8s | 46.20% | KubeAPIErrorBudgetBurn openshift-kube-apiserver Critical | 4.18 | | amd64 | ovn | gcp | ha | 143 | 0s| 0.00% | 23.2s | 51.63% | etcdHighCommitDurations openshift-etcd Warning | 4.18 | 4.18 | amd64 | ovn | azure | ha | 225 | 5m48.8s| 0.00% | 33m25.52s | 244.35% | CsvAbnormalFailedOver2Min openshift-operator-lifecycle-manager Warning | 4.18 | | amd64 | ovn | aws | ha | 321 | 0s| 0.00% | 5m20.4s | 5900.00% | etcdHighCommitDurations openshift-etcd Warning | 4.18 | | amd64 | ovn | azure | ha | 130 | 5m43s| 0.00% | 9m28s | 43.49% | ExtremelyHighIndividualControlPlaneCPU openshift-kube-apiserver Warning | 4.18 | | amd64 | ovn | aws | ha | 321 | 0s| 0.00% | 4m16s | 30376.19% | KubeAPIErrorBudgetBurn openshift-kube-apiserver Critical | 4.18 | | amd64 | ovn | azure | ha | 130 | 1m44.5s| 0.00% | 6m22.78s | 129.54% | AuditLogError openshift-kube-apiserver Warning | 4.18 | | amd64 | ovn | aws | ha | 321 | 17m54s| 0.00% | 21m48s | 4068.26% | KubeAPIErrorBudgetBurn openshift-kube-apiserver Critical | 4.18 | 4.18 | amd64 | ovn | aws | ha | 257 | 0s| 0.00% | 2m28s | 383.03%/cc @dgoodwin, @neisw, @DennisPeriquet
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.
/lgtm
[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: dgoodwin, openshift-bot
The full list of commands accepted by this bot can be found here.
The pull request process is described here
/retest-required
Remaining retests: 0 against base HEAD 90f69547de0ebd6847a4a5ca0c8c44e02ea2d7ef and 2 for PR HEAD b3fb48a286d9b1d3989331b8a3f97ca258fe956e in total
/retest-required
Remaining retests: 0 against base HEAD 2ae2adfa024b4ff3b82b19dade404273cf7cd7b2 and 2 for PR HEAD b3fb48a286d9b1d3989331b8a3f97ca258fe956e in total
Job Failure Risk Analysis for sha: b3fb48a286d9b1d3989331b8a3f97ca258fe956e
Job Name | Failure Risk |
---|---|
pull-ci-openshift-origin-master-e2e-aws-ovn-edge-zones | High [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above info This test has passed 100.00% of 4 runs on jobs ['periodic-ci-openshift-release-master-nightly-4.18-e2e-aws-ovn-edge-zones' 'periodic-ci-openshift-release-master-nightly-4.17-e2e-aws-ovn-edge-zones'] in the last 14 days. Open Bugs SNO error rate for alert/KubeAPIErrorBudgetBurn should not be at or above info |
/retest-required
Remaining retests: 0 against base HEAD 2ae2adfa024b4ff3b82b19dade404273cf7cd7b2 and 2 for PR HEAD b3fb48a286d9b1d3989331b8a3f97ca258fe956e in total
@openshift-bot: The following tests failed, say /retest
to rerun all failed tests or /retest-required
to rerun all mandatory failed tests:
Test name | Commit | Details | Required | Rerun command |
---|---|---|---|---|
ci/prow/e2e-metal-ipi-ovn-kube-apiserver-rollout | b3fb48a286d9b1d3989331b8a3f97ca258fe956e | link | false | /test e2e-metal-ipi-ovn-kube-apiserver-rollout |
ci/prow/e2e-aws-ovn-single-node-serial | b3fb48a286d9b1d3989331b8a3f97ca258fe956e | link | false | /test e2e-aws-ovn-single-node-serial |
ci/prow/e2e-aws-ovn-kube-apiserver-rollout | b3fb48a286d9b1d3989331b8a3f97ca258fe956e | link | false | /test e2e-aws-ovn-kube-apiserver-rollout |
Full PR test history. Your PR dashboard.
/retest-required
Remaining retests: 0 against base HEAD 6fdaf6b0229e812d60851ac403d6007f816205e5 and 1 for PR HEAD b3fb48a286d9b1d3989331b8a3f97ca258fe956e in total
/retest-required
Remaining retests: 0 against base HEAD 57f905381526348b1b90a8b88f7c6344022c87b1 and 2 for PR HEAD b3fb48a286d9b1d3989331b8a3f97ca258fe956e in total
[ART PR BUILD NOTIFIER]
Distgit: openshift-enterprise-tests This PR has been included in build openshift-enterprise-tests-container-v4.19.0-202411190705.p0.g9ab7580.assembly.stream.el9. All builds following this will include this PR.
disruptions Information
There were (
420
) added jobs and (0
) were removed.Comparisons were above allowed leeway of
30.00%
Note: For P99, disruptions had
199
jobs increased and733
jobs decreased.Click To Show Table
| Name | Release | From | Arch | Network | Platform | Topology | Job Results | P95 | P95 % Increase | P99 | P99 % Increase | | ---- | ------- | ---- | ---- | ------- | -------- |--------- | ----------- | --- | -------------- | --- | -------------- | | openshift-api-reused-connections | 4.18 | | amd64 | ovn | metal | ha | 387 | 6.7s| 570.00% | 23.7s | 141.59% | ingress-to-oauth-server-new-connections | 4.18 | 4.18 | arm64 | ovn | aws | ha | 106 | 1.75s| 75.00% | 3.95s | 99.49% | kube-api-new-connections | 4.17 | 4.17 | amd64 | ovn | gcp | ha | 116 | 1s| 185.71% | 2.7s | 39.18% | cache-kube-api-new-connections | 4.17 | 4.17 | amd64 | ovn | gcp | ha | 116 | 0s| 0.00% | 1.85s | 85.00% | ingress-to-console-new-connections | 4.17 | | amd64 | ovn | aws | single | 435 | 0s| 0.00% | 1.66s | 66.00% | azure-network-liveness-new-connections | 4.17 | | arm64 | ovn | aws | ha | 313 | 0s| 0.00% | 880ms | 1660.00% | ingress-to-oauth-server-reused-connections | 4.17 | 4.16 | amd64 | ovn | gcp | ha | 153 | 1s| 0.00% | 1.48s | 48.00% | cache-oauth-api-new-connections | 4.17 | 4.17 | amd64 | ovn | gcp | ha | 116 | 0s| 0.00% | 850ms | 80.85% | ci-cluster-network-liveness-new-connections | 4.17 | 4.16 | amd64 | ovn | gcp | ha | 153 | 0s| 0.00% | 3.48s | 81.25% | pod-to-service-new-connections | 4.17 | 4.16 | amd64 | ovn | aws | ha | 165 | 0s| 0.00% | 1.36s | 36.00% | openshift-api-new-connections | 4.17 | | amd64 | ovn | gcp | ha | 439 | 0s| 0.00% | 1s | 92.31% | host-to-service-new-connections | 4.17 | 4.16 | amd64 | ovn | aws | ha | 165 | 800ms| 0.00% | 1.36s | 36.00% | openshift-api-new-connections | 4.17 | | amd64 | ovn | metal | ha | 315 | 7.6s| 0.00% | 38.72s | 71.94% | oauth-api-new-connections | 4.17 | 4.16 | amd64 | ovn | gcp | ha | 153 | 0s| 0.00% | 1.48s | 48.00% | host-to-pod-new-connections | 4.17 | | s390x | ovn | libvirt | ha | 127 | 0s| 0.00% | 1s | 63.93% | azure-network-liveness-new-connections | 4.17 | 4.16 | amd64 | ovn | aws | ha | 165 | 0s| 0.00% | 54.72s | 90.00% | azure-network-liveness-new-connections | 4.17 | | heterogeneous | ovn | aws | ha | 133 | 0s| 0.00% | 2.04s | 140.00% | kube-api-reused-connections | 4.17 | | amd64 | ovn | azure | ha | 380 | 1s| 0.00% | 3s | 50.00% | host-to-pod-new-connections | 4.17 | 4.16 | amd64 | ovn | aws | ha | 165 | 1s| 0.00% | 2.72s | 36.00% | cache-openshift-api-reused-connections | 4.17 | 4.16 | amd64 | ovn | azure | ha | 108 | 1s| 0.00% | 2.93s | 46.50% | image-registry-reused-connections | 4.17 | 4.17 | amd64 | ovn | gcp | ha | 116 | 1s| 0.00% | 1.85s | 85.00% | ingress-to-oauth-server-new-connections | 4.17 | 4.17 | amd64 | ovn | gcp | ha | 116 | 1s| 0.00% | 1.85s | 85.00% | metrics-api-reused-connections | 4.17 | | amd64 | ovn | metal | ha | 334 | 2s| 0.00% | 2.67s | 111.90% | oauth-api-reused-connections | 4.17 | | amd64 | ovn | azure | ha | 380 | 1s| 0.00% | 3.42s | 71.00% | image-registry-reused-connections | 4.18 | 4.17 | amd64 | ovn | aws | ha | 1104 | 0s| 0.00% | 3.79s | 39.85% | azure-network-liveness-new-connections | 4.18 | | heterogeneous | ovn | aws | ha | 202 | 0s| 0.00% | 1m25.14s | 19250.00% | pod-to-service-new-connections | 4.18 | | amd64 | ovn | metal | compact | 104 | 0s| 0.00% | 970ms | 115.56% | service-load-balancer-with-pdb-new-connections | 4.18 | | heterogeneous | ovn | aws | ha | 202 | 1s| 0.00% | 1.99s | 38.19% | image-registry-new-connections | 4.18 | | arm64 | ovn | aws | ha | 447 | 1s| 0.00% | 2s | 70.94% | oauth-api-new-connections | 4.18 | 4.18 | amd64 | ovn | azure | ha | 545 | 1s| 0.00% | 3s | 50.00% | kube-api-new-connections | 4.18 | | heterogeneous | ovn | aws | ha | 202 | 0s| 0.00% | 1s | 127.27% | oauth-api-new-connections | 4.18 | | heterogeneous | ovn | aws | ha | 202 | 0s| 0.00% | 1s | 127.27% | ci-cluster-network-liveness-new-connections | 4.18 | 4.18 | arm64 | ovn | aws | ha | 106 | 0s| 0.00% | 27.15s | 151.86% | cache-kube-api-new-connections | 4.18 | | heterogeneous | ovn | aws | ha | 202 | 0s| 0.00% | 1.98s | 350.00% | service-load-balancer-with-pdb-new-connections | 4.18 | | arm64 | ovn | aws | ha | 447 | 0s| 0.00% | 1.54s | 54.00% | host-to-service-new-connections | 4.18 | | amd64 | ovn | metal | ha | 363 | 0s| 0.00% | 1s | 426.32% | pod-to-pod-new-connections | 4.18 | 4.18 | amd64 | ovn | aws | single | 534 | 0s| 0.00% | 1s | 51.52% | service-load-balancer-with-pdb-new-connections | 4.18 | | amd64 | ovn | azure | ha | 461 | 0s| 0.00% | 400ms | 566.67% | pod-to-pod-reused-connections | 4.18 | 4.18 | amd64 | ovn | aws | single | 534 | 0s| 0.00% | 1s | 51.52% | kube-api-reused-connections | 4.18 | 4.17 | amd64 | ovn | azure | ha | 980 | 2s| 0.00% | 5.21s | 30.25% | host-to-service-new-connections | 4.18 | | amd64 | ovn | metal | compact | 104 | 0s| 0.00% | 970ms | 94.00% | cache-kube-api-reused-connections | 4.18 | | amd64 | ovn | vsphere | ha | 783 | 0s| 0.00% | 1s | 108.33% | host-to-pod-new-connections | 4.18 | | s390x | ovn | libvirt | ha | 185 | 0s| 0.00% | 1s | 156.41%alerts Information
There were (
1926
) added jobs and (30
) were removed.Comparisons were above allowed leeway of
30.00%
Note: For P99, alerts had
47
jobs increased and299
jobs decreased.Click To Show Table
| Name | Release | From | Arch | Network | Platform | Topology | Job Results | P95 | P95 % Increase | P99 | P99 % Increase | | ---- | ------- | ---- | ---- | ------- | -------- |--------- | ----------- | --- | -------------- | --- | -------------- | | TechPreviewNoUpgrade openshift-kube-apiserver-operator Warning | 4.18 | | amd64 | ovn | metal | ha | 298 | 1h16m5.8s| 32.92% | 1h20m33.02s | 30.64% | NodeSystemSaturation openshift-monitoring Warning | 4.18 | | amd64 | ovn | aws | single | 122 | 6m7.5s| 101.92% | 8m52.74s | 93.98% | HighOverallControlPlaneCPU openshift-kube-apiserver Warning | 4.18 | | amd64 | ovn | gcp | ha | 143 | 2m53s| 198.28% | 4m0.8s | 83.12% | NodeSystemSaturation openshift-monitoring Warning | 4.18 | 4.18 | amd64 | ovn | aws | single | 185 | 2m45.6s| 88.18% | 3m34.96s | 45.24% | HighOverallControlPlaneCPU openshift-kube-apiserver Warning | 4.18 | 4.18 | amd64 | ovn | gcp | ha | 274 | 1m28s| 214.29% | 1m58s | 103.45% | CannotRetrieveUpdates openshift-cluster-version Warning | 4.17 | | amd64 | ovn | aws | ha | 128 | 0s| 0.00% | 14m47.68s | 1205.41% | KubeJobFailed openshift-insights Warning | 4.17 | | amd64 | ovn | aws | ha | 128 | 59m29.9s| 0.00% | 2h51m29.86s | 66.73% | KubeAPIErrorBudgetBurn openshift-kube-apiserver Critical | 4.17 | | amd64 | ovn | aws | ha | 128 | 0s| 0.00% | 1m41.8s | 2093.97% | TargetDown openshift-ingress Warning | 4.17 | | amd64 | ovn | aws | ha | 128 | 0s| 0.00% | 31m8.8s | 818.96% | KubeletPlegDurationHigh kube-system Warning | 4.18 | | amd64 | ovn | gcp | ha | 143 | 0s| 0.00% | 58s | 360.32% | KubeJobFailed openshift-insights Warning | 4.18 | | amd64 | ovn | aws | ha | 321 | 1h43m6s| 0.00% | 3h12m16.8s | 46.20% | KubeAPIErrorBudgetBurn openshift-kube-apiserver Critical | 4.18 | | amd64 | ovn | gcp | ha | 143 | 0s| 0.00% | 23.2s | 51.63% | etcdHighCommitDurations openshift-etcd Warning | 4.18 | 4.18 | amd64 | ovn | azure | ha | 225 | 5m48.8s| 0.00% | 33m25.52s | 244.35% | CsvAbnormalFailedOver2Min openshift-operator-lifecycle-manager Warning | 4.18 | | amd64 | ovn | aws | ha | 321 | 0s| 0.00% | 5m20.4s | 5900.00% | etcdHighCommitDurations openshift-etcd Warning | 4.18 | | amd64 | ovn | azure | ha | 130 | 5m43s| 0.00% | 9m28s | 43.49% | ExtremelyHighIndividualControlPlaneCPU openshift-kube-apiserver Warning | 4.18 | | amd64 | ovn | aws | ha | 321 | 0s| 0.00% | 4m16s | 30376.19% | KubeAPIErrorBudgetBurn openshift-kube-apiserver Critical | 4.18 | | amd64 | ovn | azure | ha | 130 | 1m44.5s| 0.00% | 6m22.78s | 129.54% | AuditLogError openshift-kube-apiserver Warning | 4.18 | | amd64 | ovn | aws | ha | 321 | 17m54s| 0.00% | 21m48s | 4068.26% | KubeAPIErrorBudgetBurn openshift-kube-apiserver Critical | 4.18 | 4.18 | amd64 | ovn | aws | ha | 257 | 0s| 0.00% | 2m28s | 383.03%/cc @dgoodwin, @neisw, @DennisPeriquet