kubernetes-sigs / gateway-api

Repository for the next iteration of composite service (e.g. Ingress) and load balancing APIs.
https://gateway-api.sigs.k8s.io
Apache License 2.0
1.67k stars 437 forks source link

Add hook points support in conformance test suite #3131

Open shawnh2 opened 3 weeks ago

shawnh2 commented 3 weeks ago

What would you like to be added:

Add the hook point support for current conformance test suite.

For example, like AfterTestFail hook point, developers can attach various commands to this hook point, and programs that attach to this hook point wil only be executed if one conformance test is failed.

Why this is needed:

This will be super helpful for developers to debug the failed conformance test.

Sometimes the error trace reported by conformance test is not very useful. For example, we'd like to see the status/logs of specific resources (by attaching kubectl cmd to the hook which I described above could ease the pain), even some other configs that can be retrieved by other commands.

k8s-ci-robot commented 3 weeks ago

@mlavacca: The label(s) /label needs-triage cannot be applied. These labels are supported: api-review, tide/merge-method-merge, tide/merge-method-rebase, tide/merge-method-squash, team/katacoda, refactor. Is this label configured under labels -> additional_labels or labels -> restricted_labels in plugin.yaml?

In response to [this](https://github.com/kubernetes-sigs/gateway-api/issues/3131#issuecomment-2147411123): >/label needs-triage Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.
k8s-ci-robot commented 3 weeks ago

@mlavacca: The label(s) triage/needs cannot be applied, because the repository doesn't have them.

In response to [this](https://github.com/kubernetes-sigs/gateway-api/issues/3131#issuecomment-2147415530): >/triage needs Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.
mlavacca commented 3 weeks ago

/triage needs-triage

k8s-ci-robot commented 3 weeks ago

@mlavacca: The label(s) triage/needs-triage cannot be applied, because the repository doesn't have them.

In response to [this](https://github.com/kubernetes-sigs/gateway-api/issues/3131#issuecomment-2147489112): >/triage needs-triage Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.
kflynn commented 3 weeks ago

This sounds like a good idea, yeah. Making it easier for developers to get value from the conformance tests, whether or not they're "part of the Gateway API", sounds like a Good Thing™. 🙂

robscott commented 3 weeks ago

Discussed in today's community meeting, this sounds like a great idea, thanks @shawnh2!

/triage accepted

shawnh2 commented 3 weeks ago

Assign myself, since I've been working on something that is quite similar to this issue as well. I will post the initial design of this here once it's done.

/assign