search

kubernetes / minikube

Run Kubernetes locally
https://minikube.sigs.k8s.io/
Apache License 2.0
29.5k stars 4.89k forks source link

testing: Move bulk of GCP-Auth addon test into subtest #19716

Closed spowelljr closed 1 month ago

spowelljr commented 1 month ago

Simply moving all of the GCP-Auth addon test that's not already in a subtest into a sub test so failures are properly displayed in gopogh, part of https://github.com/kubernetes/minikube/issues/19714.

spowelljr commented 1 month ago

/ok-to-test

minikube-pr-bot commented 1 month ago

kvm2 driver with docker runtime

+-------------------+----------+---------------------+
|      COMMAND      | MINIKUBE | MINIKUBE (PR 19716) |
+-------------------+----------+---------------------+
| minikube start    | 50.6s    | 50.1s               |
| ⚠️  enable ingress | 15.9s    | 27.6s ⚠️             |
+-------------------+----------+---------------------+
Times for minikube (PR 19716) ingress: 16.1s 15.1s 16.6s 15.6s 74.6s Times for minikube ingress: 16.1s 15.6s 16.0s 16.6s 15.6s Times for minikube start: 50.6s 49.8s 49.4s 51.9s 51.3s Times for minikube (PR 19716) start: 50.0s 50.0s 51.3s 51.2s 48.1s

docker driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 19716) |
+----------------+----------+---------------------+
| minikube start | 22.7s    | 23.7s               |
| enable ingress | 12.6s    | 11.4s               |
+----------------+----------+---------------------+
Times for minikube start: 23.4s 23.3s 20.9s 24.2s 21.6s Times for minikube (PR 19716) start: 23.5s 22.0s 24.5s 24.7s 23.9s Times for minikube ingress: 13.3s 12.8s 12.3s 12.3s 12.3s Times for minikube (PR 19716) ingress: 12.8s 10.3s 10.8s 12.8s 10.3s

docker driver with containerd runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 19716) |
+----------------+----------+---------------------+
| minikube start | 20.5s    | 21.6s               |
| enable ingress | 26.2s    | 26.0s               |
+----------------+----------+---------------------+
Times for minikube ingress: 22.8s 22.8s 23.8s 38.9s 22.8s Times for minikube (PR 19716) ingress: 22.8s 38.8s 22.8s 22.8s 22.8s Times for minikube start: 19.6s 20.4s 19.4s 20.2s 22.7s Times for minikube (PR 19716) start: 20.1s 23.0s 21.1s 21.2s 22.5s
minikube-pr-bot commented 1 month ago

Here are the number of top 10 failed tests in each environments with lowest flake rate.

Environment Test Name Flake Rate
Docker_Linux_crio_arm64 (4 failed) TestAddons/serial/GCPAuth/PullSecret(gopogh) Unknown
Docker_Linux_docker_arm64 (2 failed) TestAddons/serial/GCPAuth/PullSecret(gopogh) Unknown
none_Linux (2 failed) TestAddons/serial/GCPAuth/PullSecret(gopogh) Unknown
KVM_Linux (2 failed) TestAddons/serial/GCPAuth/PullSecret(gopogh) Unknown
Docker_Cloud_Shell (7 failed) TestAddons/serial/GCPAuth/PullSecret(gopogh) Unknown
Docker_Linux_crio (4 failed) TestAddons/serial/GCPAuth/PullSecret(gopogh) Unknown
Docker_Linux (2 failed) TestAddons/serial/GCPAuth/PullSecret(gopogh) Unknown

Besides the following environments also have failed tests:

To see the flake rates of all tests by environment, click here.

minikube-pr-bot commented 1 month ago

kvm2 driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 19716) |
+----------------+----------+---------------------+
| minikube start | 50.9s    | 52.4s               |
| enable ingress | 15.5s    | 15.3s               |
+----------------+----------+---------------------+
Times for minikube start: 49.4s 50.4s 52.5s 52.1s 50.2s Times for minikube (PR 19716) start: 55.9s 50.1s 51.7s 52.6s 51.6s Times for minikube ingress: 16.0s 15.0s 14.5s 16.0s 16.0s Times for minikube (PR 19716) ingress: 15.5s 14.6s 15.6s 15.1s 15.6s

docker driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 19716) |
+----------------+----------+---------------------+
| minikube start | 22.4s    | 22.2s               |
| enable ingress | 12.5s    | 11.8s               |
+----------------+----------+---------------------+
Times for minikube start: 21.3s 21.1s 24.1s 21.0s 24.7s Times for minikube (PR 19716) start: 21.0s 23.8s 21.0s 24.4s 20.9s Times for minikube ingress: 12.3s 12.8s 12.8s 12.3s 12.3s Times for minikube (PR 19716) ingress: 10.8s 10.3s 12.3s 12.8s 12.8s

docker driver with containerd runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 19716) |
+----------------+----------+---------------------+
| minikube start | 20.8s    | 21.8s               |
| enable ingress | 22.9s    | 26.1s               |
+----------------+----------+---------------------+
Times for minikube start: 19.7s 22.6s 20.3s 21.0s 20.5s Times for minikube (PR 19716) start: 22.7s 23.5s 22.8s 19.5s 20.5s Times for minikube ingress: 22.8s 22.8s 22.8s 22.8s 23.3s Times for minikube (PR 19716) ingress: 22.8s 22.8s 22.8s 39.3s 22.8s
minikube-pr-bot commented 1 month ago

kvm2 driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 19716) |
+----------------+----------+---------------------+
| minikube start | 48.4s    | 49.2s               |
| enable ingress | 15.1s    | 15.3s               |
+----------------+----------+---------------------+
Times for minikube start: 47.1s 50.1s 47.5s 47.4s 49.8s Times for minikube (PR 19716) start: 46.3s 50.2s 47.9s 52.6s 48.9s Times for minikube ingress: 15.5s 14.5s 15.5s 14.5s 15.5s Times for minikube (PR 19716) ingress: 15.0s 15.5s 15.0s 15.0s 16.0s

docker driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 19716) |
+----------------+----------+---------------------+
| minikube start | 22.4s    | 21.9s               |
| enable ingress | 12.8s    | 13.2s               |
+----------------+----------+---------------------+
Times for minikube ingress: 13.3s 13.3s 12.8s 13.3s 11.3s Times for minikube (PR 19716) ingress: 12.3s 13.3s 13.3s 13.3s 13.8s Times for minikube start: 20.8s 23.4s 23.3s 20.8s 23.7s Times for minikube (PR 19716) start: 20.9s 21.0s 20.3s 23.4s 23.6s

docker driver with containerd runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 19716) |
+----------------+----------+---------------------+
| minikube start | 21.3s    | 21.8s               |
| enable ingress | 26.4s    | 24.8s               |
+----------------+----------+---------------------+
Times for minikube (PR 19716) start: 20.9s 23.0s 22.0s 22.7s 20.4s Times for minikube start: 22.6s 22.6s 20.1s 20.4s 20.9s Times for minikube ingress: 23.8s 22.8s 39.8s 22.8s 22.8s Times for minikube (PR 19716) ingress: 23.8s 24.3s 23.3s 23.8s 28.8s
minikube-pr-bot commented 1 month ago

Here are the number of top 10 failed tests in each environments with lowest flake rate.

Environment Test Name Flake Rate
Docker_Linux (1 failed) TestAddons/serial/GCPAuth/PullSecret(gopogh) Unknown
Docker_Linux_docker_arm64 (1 failed) TestAddons/serial/GCPAuth/PullSecret(gopogh) Unknown
KVM_Linux (1 failed) TestAddons/serial/GCPAuth/PullSecret(gopogh) Unknown
none_Linux (1 failed) TestAddons/serial/GCPAuth/PullSecret(gopogh) Unknown
Docker_Linux_crio_arm64 (3 failed) TestAddons/serial/GCPAuth/PullSecret(gopogh) Unknown
Docker_Cloud_Shell (6 failed) TestAddons/serial/GCPAuth/PullSecret(gopogh) Unknown
Docker_Linux_crio (3 failed) TestAddons/serial/GCPAuth/PullSecret(gopogh) Unknown

Besides the following environments also have failed tests:

To see the flake rates of all tests by environment, click here.

minikube-pr-bot commented 1 month ago

Here are the number of top 10 failed tests in each environments with lowest flake rate.

Environment Test Name Flake Rate
Docker_Linux (1 failed) TestAddons/serial/GCPAuth/PullSecret(gopogh) Unknown
Docker_Linux_crio (3 failed) TestAddons/serial/GCPAuth/PullSecret(gopogh) Unknown
KVM_Linux (1 failed) TestAddons/serial/GCPAuth/PullSecret(gopogh) Unknown
none_Linux (1 failed) TestAddons/serial/GCPAuth/PullSecret(gopogh) Unknown
Docker_Linux_crio_arm64 (3 failed) TestAddons/serial/GCPAuth/PullSecret(gopogh) Unknown
Docker_Linux_docker_arm64 (1 failed) TestAddons/serial/GCPAuth/PullSecret(gopogh) Unknown

Besides the following environments also have failed tests:

To see the flake rates of all tests by environment, click here.

medyagh commented 1 month ago

/lgtm

k8s-ci-robot commented 1 month ago

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: medyagh, spowelljr

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/kubernetes/minikube/blob/master/OWNERS)~~ [medyagh,spowelljr] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment