e2e-k8s.sh: support --ginkgo.label-filter

pohly commented 2 months ago

The label filter query is more expressive (logical operations) and readable (no regexp unless absolutely required). Such a query can be combined with focus + skip, but in practice a single label filter can replace both of those and is easier to understand.

Kubernetes has supported ginkgo v2 and thus --label-filter since v1.25.0. This makes it safe to pass that command line flag unconditionally when invoking the E2E suite. However, actual labels were only added in Kubernetes 1.29.

Note: I have not tested this yet. If we are confident that it'll work, then we can merge and I'll start using it in a small job (most likely DRA) once it is available.

pohly commented 2 months ago

Is my understanding correct that this change will be usable right after merging, i.e. without a formal kind release?

aojea commented 3 weeks ago

Is my understanding correct that this change will be usable right after merging, i.e. without a formal kind release?

yes

@pohly @onsi this output looks weird, Will run 2144 of 7407 specs but it Ran 875 of 7407 Specs

/home/prow/go/src/k8s.io/kubernetes/_output/bin/ginkgo --nodes=25 --poll-progress-after=60m --poll-progress-interval=5m --source-root=/home/prow/go/src/k8s.io/kubernetes/cluster/.. --timeout=24h --flake-attempts=1 --no-color /home/prow/go/src/k8s.io/kubernetes/_output/bin/e2e.test -- --kubeconfig=/root/.kube/kind-test-config --host=https://127.0.0.1:41971/ --provider=skeleton --gce-project= --gce-zone= --gce-region= --gce-multizone=false --gke-cluster= --kube-master= --cluster-tag= --cloud-config-file= --repo-root=/home/prow/go/src/k8s.io/kubernetes/cluster/.. --node-instance-group= --prefix=e2e --network=e2e --node-tag= --master-tag= --docker-config-file= --dns-domain=cluster.local --prepull-images=false --report-complete-ginkgo --report-complete-junit --provider=skeleton --num-nodes=2 --ginkgo.focus=. '--ginkgo.skip=\[Serial\]|\[Slow\]|\[Disruptive\]|\[Flaky\]|\[Feature:.+\]|PodSecurityPolicy|LoadBalancer|load.balancer|Simple.pod.should.support.exec.through.an.HTTP.proxy|subPath.should.support.existing' --ginkgo.label-filter= --report-dir=/logs/artifacts --disable-log-dump=true
Running Suite: Kubernetes e2e suite - /home/prow/go/src/k8s.io/kubernetes/_output/bin
=====================================================================================
Random Seed: 1713281512 - will randomize all specs
Will run 2144 of 7407 specs
Running in parallel across 25 processes
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
... snipped ...
Ran 875 of 7407 Specs in 924.114 seconds
SUCCESS! -- 875 Passed | 0 Failed | 0 Pending | 6532 Skipped

onsi commented 3 weeks ago

@aojea i don’t know if y’all also call Skip() in some specs at runtime. That would explain the discrepancy.

pohly commented 3 weeks ago

We do a lot of runtime skipping. I think that explains it.

aojea commented 3 weeks ago

one last comment and we are good to go https://github.com/kubernetes-sigs/kind/pull/3582/files#r1635882841

aojea commented 3 weeks ago

/lgtm /approve

/cc: @BenTheElder

k8s-ci-robot commented 3 weeks ago

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: aojea, pohly

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/kubernetes-sigs/kind/blob/main/OWNERS)~~ [aojea] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment

aojea commented 3 weeks ago

we should also think in phasing this: 1.31 start the transition to labels and deprecation focus and skip , warn on test-infra to use filters

1.32 phase out and forbid focus and skip in test-infra with a. linter

kubernetes-sigs / kind

e2e-k8s.sh: support --ginkgo.label-filter #3582