Add e2e to ensure the Kubernetes namespace controller is functional

kcp-dev / kcp

Kubernetes-like control planes for form-factors and use-cases beyond Kubernetes and container workloads.

https://kcp.io

Apache License 2.0

2.35k stars 381 forks source link

Add e2e to ensure the Kubernetes namespace controller is functional #400

Closed ncdc closed 4 months ago

ncdc commented 2 years ago

The Kubernetes namespace controller is responsible for deleting all resources from a namespace when the namespace is deleted. Because kcp has resources in multiple logical clusters, we need to make sure the namespace controller is able to delete resources from namespaces from multiple logical clusters. It's possible we might accidentally break things in the future as we refactor & improve kcp's code, so let's make sure we have an end to end test that will catch if we break the namespace controller.

marun commented 2 years ago

Does this imply running a subset of upstream kube's compliance e2e or would this testing be net-new?

ncdc commented 2 years ago

Long-term, it probably implies carving out a subset of "generic control plane" conformance tests from upstream. But if upstream never has multiple logical clusters, I think we'll still need to create kcp variants to ensure things work in a multi-logical cluster setup. WDYT?

sttts commented 2 years ago

These cross-workspace controllers with dynamic RESTMapper are deeply broken right now. We cannot poll discovery for that. We have to come up with something else, e.g. using discovery for the native resources, and an CRD informer for the rest.

ncdc commented 2 years ago

@sttts could you please elaborate on the discovery issues?

sttts commented 2 years ago

We have 60s poll interval updating a RESTMapper in a couple of places in kube-apiserver (for admission I guess) and in kube-controller-manager for GC, namespace and quota. That RESTMapper is used to iterate over all known types and its critical for consistency.

Types change by workspace (different CRDs in different workspaces). So we need a different resource list for each. Doing that with polling is not a good idea. Something that we can watch would be much better.

ncdc commented 2 years ago

Clearing milestone to re-triage

sttts commented 2 years ago

cc @qiujian16

kcp-ci-bot commented 6 months ago

Issues go stale after 90d of inactivity. After a furter 30 days, they will turn rotten. Mark the issue as fresh with /remove-lifecycle stale.

If this issue is safe to close now please do so with /close.

/lifecycle stale

kcp-ci-bot commented 5 months ago

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

/lifecycle rotten

kcp-ci-bot commented 4 months ago

Rotten issues close after 30d of inactivity. Reopen the issue with /reopen. Mark the issue as fresh with /remove-lifecycle rotten.

/close

kcp-ci-bot commented 4 months ago

@kcp-ci-bot: Closing this issue.

In response to [this](https://github.com/kcp-dev/kcp/issues/400#issuecomment-2161559811): >Rotten issues close after 30d of inactivity. >Reopen the issue with `/reopen`. >Mark the issue as fresh with `/remove-lifecycle rotten`. > >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.