kubernetes-sigs / ingress-controller-conformance

Repository for a compliance specification of ingress-controllers.
Apache License 2.0
43 stars 36 forks source link

Consider dropping the test hierarchy #15

Closed jpeach closed 4 years ago

jpeach commented 4 years ago

The conformance tests are organized in a hierarchy of check.Check structs. This makes initialization fairly verbose, but I'm not sure that the hierarchy is all that useful.

If we drop the hierarchy, we could replace all the initialization, with something like this:

var _ = checks.Register(check.Check{
    Name:        "path-rules-prefix-invalid-characters",
    Description: "Ingress with prefix path rule with invalid characters are ignored",
    Run: func(check *Check, config Config) (success bool, err error) {
        ....
        },
})

Alternatively, we could use reflection to find tests. Maybe we could add a regex argument to the verify subcommand to replace the grouping behavior of the current hierarchy.

bowei commented 4 years ago

@aledbf I think has an interesting framework here that uses the sonabuoy compatible tests:

https://github.com/aledbf/ingress-conformance

maybe we can borrow something from this?

jpeach commented 4 years ago

I think that the relative simplicity of the framework here is nice. The obvious downside is that of maintaining a separate framework, but maybe that won't be too painful if we keep it simple :)

fejta-bot commented 4 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

bowei commented 4 years ago

/remove-lifecycle stale

aledbf commented 4 years ago

Closing. The repository is already using BDD (a similar approach to aledbf/ingress-conformance)

/close

k8s-ci-robot commented 4 years ago

@aledbf: Closing this issue.

In response to [this](https://github.com/kubernetes-sigs/ingress-controller-conformance/issues/15#issuecomment-659128417): >Closing. The repository is already using BDD (a similar approach to aledbf/ingress-conformance) > >/close > Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.