ApplySet: More integration with core components of Kubernetes.

koba1t commented 9 months ago

I believe ApplySet is a beneficial function, and many users will use about an alternative for the prune option. I think it would be more beneficial to combine Kubernetes code in the next implementation. So, I'd like to offer some feature proposals.

1. Server-side pruning

What would you like to be added:

Execute the pruning function on the server side rather than the client side.

Why is this needed:

I believe it would be more effective to execute the pruning function on the server side rather than the client side. When pruning is performed on the client side, it can result in varying behavior depending on the pruning logic used by different versions of the tool and any tool that manages the same resources. I read the alternative section of OwnerRef in Proposal and I understand we have any issues if we use OwnerRef for pruning some resources. But we can consider more to a way to adopt Server-side pruning.

2. Define any resource type for cluster-wide or cross-namespace resources

What would you like to be added:

Add a new type of resource or assign any existing cluster-wide resource for ApplySet parents when managing cluster-wide resources or cross-namespace resources.

Why is this needed:

I think we need any type of resource that can be ApplySet parents for cluster-wide or cross-namespace resources for unified usability. My understanding is that in many cases, Kubernetes is currently used by platform users to manage any namespace resource in their namespace and cluster admin management cluster-wide and any admin's namespace resources. I think we need more consideration of usability for this case.

additional

If you think my idea had any misunderstanding or if I missed any discussion related to this, please let me know the details, and I reconsider this proposal. Additionally, If you need more help writing Kep or the implementation function with the ApplySet, I want to help, and please tell me what I should do.

k8s-ci-robot commented 9 months ago

This issue is currently awaiting triage.

SIG CLI takes a lead on issue triage for this repo, but any Kubernetes member can accept issues by applying the triage/accepted label.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.

koba1t commented 8 months ago

/assign @justinsb @KnVerey

justinsb commented 7 months ago

Thanks @koba1t - I share your enthusiasm for ApplySet :-)

The issue with more server-side support is that it's difficult to find a good design; but of course if you can find one then I think people would be open to it. Some of the things that are challenging:

ownerRefs are a little limited: a namespaced resource cannot be the owner of a cluster-scoped resource, and cross-namespace ownerRefs are disallowed: https://github.com/kubernetes/kubernetes/issues/94631. Maybe we could argue that if you have a multi-namespaced package you should have a cluster-scoped parent etc; TBH I think that's probably good guidance for end-users. But we do have to balance that against a desire to support the existing pruning behaviours, even if they are suboptimal - if we don't support all the same use-cases, then it's hard to move away from the existing prune.
I don't think there's a nice way to accelerate ownerRef queries, so we might have to use labels as well so that the "discovery" queries are not highly inefficient. (?) We do want to be able to "preview" the results from an apply in terms of pruning.

Maybe one thing we could do is to support an additional option where we also add ownerRefs? And if your applyset spans namespaces etc then that would be an error if --with-ownerrefs is specified.

We could also address this by adding acceleration for a field selector on ownerRef: https://kubernetes.io/docs/concepts/overview/working-with-objects/field-selectors/ I think there's a reasonable case to be made, but it would need buy-in from apimachinery.

For the second proposal, creating a well-known parent object, I think this makes a lot of sense. I think it could just be a CRD, though maybe if we created a built-in type then we could add more server-side behaviour. The big downside of a built-in type is that it will take much longer than a CRD, particularly if we want to add custom behaviours.

Another reason we wanted to support arbitrary objects as parents is because we want to support other tooling, and we thought that requiring a new parent object would make it harder for them to adopt (vs just adding a few labels to helm and friends, which should be a relatively simple PR)

But I do think this idea has merit, it isn't that we're opposed (I'm not opposed at least!), it's just that we didn't think it was required and we thought that different members of the community (e.g. helm) might want to use their own objects as the parent.

If you wanted to propose some CRDs like Package and ClusterPackage, that sgtm! I think we would have to figure out where they live though (e.g. in the kubectl repo, in kubernetes/kubernetes or elsewhere - I can imagine some pushback!) Also I'm not sure whether only doing a CRD would unblock anything for you, or whether you would want some special handling in kube-apiserver. Having a good cluster-scoped CRD would be nice for ownerRefs I guess, so that would be a clear and easy win.

Also cc @johnbelamaric - john has also suggested that we should have done more server-side functionality here, to enforce more things than we can with client-side functionality.

k8s-triage-robot commented 4 months ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue as fresh with /remove-lifecycle stale
Close this issue with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

koba1t commented 4 months ago

/remove-lifecycle stale

wiktor-k commented 3 months ago

Hello :wave:

Just to add to what you've written I'm wondering how do Server-Side Apply (https://kubernetes.io/docs/reference/using-api/server-side-apply/) and ApplySets (https://kubernetes.io/blog/2023/05/09/introducing-kubectl-applyset-pruning/) interact?

My use case is having an atomic update over multiple resources, basically, asking the server to either apply my entire collection of yamls in a namespace or fail with an error without updating any of it. And it seems like there's an some overlap with these two, or maybe I'm holding it wrong :sweat_smile:

k8s-triage-robot commented 1 week ago