--flake-attempts flag need not be removed from ginkgo tester

[Rajalakshmi-Girish]

After the change https://github.com/kubernetes-sigs/kubetest2/pull/251, seems the flag --flakeattempts have been removed.

Our internal CI job that uses kubetest2 ginkgo tester has been failing with below error:

Error: unknown flag: --flake-attempts
Usage of /home/prow/go/bin/kubetest2-tester-ginkgo:
      --alsologtostderr                  log to standard error as well as files
      --env strings                      List of env variables to pass to ginkgo libraries (default [])
      --focus-regex string               Regular expression of jobs to focus on.
      --ginkgo-args string               Additional arguments supported by the ginkgo binary.
  -h, --help                             
      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
      --log_dir string                   If non-empty, write log files in this directory
      --logtostderr                      log to standard error instead of files
      --parallel int                     Run this many tests in parallel at once. (default 1)
      --skip-regex string                Regular expression of jobs to skip.
      --stderrthreshold severity         logs at or above this threshold go to stderr (default 2)
      --test-args string                 Additional arguments supported by the e2e test framework (https://godoc.org/k8s.io/kubernetes/test/e2e/framework#TestContextType).
      --test-package-dir string          The directory in the bucket which represents the type of release. Default to the release directory. (default "release")
      --test-package-marker string       The version marker in the directory containing the package version to download when unspecified. Defaults to latest.txt. (default "latest.txt")
      --test-package-url string          The url to download a kubernetes test package from. (default "https://dl.k8s.io")
      --test-package-version string      The ginkgo tester uses a test package made during the kubernetes build. The tester downloads this test package from one of the release tars published to the Release bucket. Defaults to latest. visit https://kubernetes.io/releases/ to find release names. Example: v1.20.0-alpha.0
      --timeout duration                 How long (in golang duration format) to wait for ginkgo tests to complete. (default 24h0m0s)
      --use-binaries-from-path           Look for binaries in the $PATH instead of extracting from tars downloaded from GCS.
      --use-built-binaries               Look for binaries in _rundir/$KUBETEST2_RUN_DIR instead of extracting from tars downloaded from GCS.
  -v, --v Level                          log level for V logs
      --vmodule moduleSpec               comma-separated list of pattern=N settings for file-filtered logging

Expected Result: The ginkgo tester should be allowed to use --flake-attempts flag as even when the test-package is downloaded from dl.k8s.io, we see this flag present with ginkgo binary.

[root@raji-workspace2 ~]# wget https://dl.k8s.io/ci/v1.30.0-alpha.0.320+b46e436e18b931/kubernetes-test-linux-ppc64le.tar.gz
--2023-12-19 08:38:57--  https://dl.k8s.io/ci/v1.30.0-alpha.0.320+b46e436e18b931/kubernetes-test-linux-ppc64le.tar.gz
Resolving dl.k8s.io (dl.k8s.io)... 34.107.204.206, 2600:1901:0:26f3::
Connecting to dl.k8s.io (dl.k8s.io)|34.107.204.206|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://storage.googleapis.com/k8s-release-dev/ci/v1.30.0-alpha.0.320+b46e436e18b931/kubernetes-test-linux-ppc64le.tar.gz [following]
--2023-12-19 08:38:58--  https://storage.googleapis.com/k8s-release-dev/ci/v1.30.0-alpha.0.320+b46e436e18b931/kubernetes-test-linux-ppc64le.tar.gz
Resolving storage.googleapis.com (storage.googleapis.com)... 142.250.138.207, 142.250.115.207, 142.250.114.207, ...
Connecting to storage.googleapis.com (storage.googleapis.com)|142.250.138.207|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 107774829 (103M) [application/x-tar]
Saving to: ‘kubernetes-test-linux-ppc64le.tar.gz.1’

kubernetes-test-linux-ppc64le.tar.gz.1  100%[============================================================================>] 102.78M  10.5MB/s    in 11s

2023-12-19 08:39:09 (9.79 MB/s) - ‘kubernetes-test-linux-ppc64le.tar.gz.1’ saved [107774829/107774829]

[root@raji-workspace2 ~]#  mkdir untar
[root@raji-workspace2 ~]# tar -C ./untar/ -xzf kubernetes-test-linux-ppc64le.tar.gz.1
[root@raji-workspace2 ~]# ls untar/
kubernetes
[root@raji-workspace2 ~]# untar/kubernetes/test/bin/ginkgo help run | grep flake
  --flake-attempts [int] (default: 0 - failed tests are not retried)
[root@raji-workspace2 ~]#

[Rajalakshmi-Girish]

@upodroid please confirm if this is a valid issue.

[Rajalakshmi-Girish]

It seems the--flaky-attempts flag was removed from the ginkgo tester as Kubernetes has zero flake policy. @upodroid suggested using --ginkgo-args flag if we want to have flake attempts.

[upodroid]

The Kubernetes project has zero flake policy and this flag will no longer be exposed directly via ginkgo tester.

legacy behaviour can be maintained by adding this flag --ginkgo-args="--flaky-attempts=2"

https://groups.google.com/g/kubernetes-sig-testing/c/bvqV6I8iF28/m/SHswIE70AAAJ

https://github.com/kubernetes/community/blob/master/contributors/devel/sig-testing/flaky-tests.md