search

kubernetes-sigs / kubetest2

Kubetest2 is the framework for launching and running end-to-end tests on Kubernetes.
Apache License 2.0
326 stars 105 forks source link

Ginkgo v2 flag changes break testing older k8s versions #204

Closed rifelpet closed 1 year ago

rifelpet commented 1 year ago

kOps uses kubetest2 to test all supported k8s versions. The flag changes from upgrading to ginkgo v2 (https://github.com/kubernetes-sigs/kubetest2/pull/200) impact all usage of kubetest2. Because ginkgo was only upgraded to v2 in k/k's master branch, older k8s versions still use ginkgo v1. This inconsistency results in failures to test older k8s versions.

example with k8s 1.22 from https://github.com/kubernetes/kops/pull/14061

     -------------------------------------------------------------------
    |                                                                   |
    |  Ginkgo timed out waiting for all parallel nodes to report back!  |
    |                                                                   |
     -------------------------------------------------------------------

 e2e timed out. path: ./_rundir/d40e3fa1-0f35-11ed-80a5-ea2e797f9e54
[1] flag provided but not defined: -ginkgo.flake-attempts
[1] Usage of /home/prow/go/src/k8s.io/kops/_rundir/d40e3fa1-0f35-11ed-80a5-ea2e797f9e54/e2e.test: 
...

Can we make kubetest2's ginkgo flag changes conditional based on the k8s version being tested? Or what is the expected path forward to use the latest kubetest2 while testing older supported k8s versions?

BenTheElder commented 1 year ago

https://github.com/kubernetes-sigs/kubetest2/pull/201#issuecomment-1197509112

BenTheElder commented 1 year ago

short term: consider pinning the version you use: https://github.com/kubernetes-sigs/kubetest2/issues/17#issuecomment-1194994903

rifelpet commented 1 year ago

Confirmed #206 fixed the issue for kops 👍🏻