search

kubernetes-sigs / cli-utils

This repo contains binaries that built from libraries in cli-runtime.
Apache License 2.0
158 stars 78 forks source link

Feature Request: Move ResourceGroup CRD from kpt to cli-utils #364

Open ash2k opened 3 years ago

ash2k commented 3 years ago

https://github.com/GoogleContainerTools/kpt/blob/master/docs/ROADMAP.md#4-live-apply says there are some issues with ConfigMap as an inventory object. Are they documented somewhere? I've found https://github.com/GoogleContainerTools/kpt/issues/708, but there is no explanation.

As a cli-utils user, I'd like to avoid any issues with ConfigMap and use the ResourceGroup CR. I wonder if it would be possible to make it part of cli-utils?

mortent commented 3 years ago

@ash2k So the change from using a CRD instead of a ConfigMap to store the inventory object actually somewhat conflates two issues, only one of which is actually related to the use of a ConfigMap.

First, the challenges that are specific to a ConfigMap were:

Somewhat separately, the first implementation of the inventory created a new inventory object on every apply and then removed old ones as they were no longer needed. This lead to some additional complexity so the current algorithm just updates the inventory object in place. There are still some remnants of the original approach in the code, but we eventually want to get rid of it.

I will look into whether the ResourceGroup CRD can be made available somewhere. We do want to allow users to chose the resource type for storing the inventory, which is why the support for ResourceGroup is implemented in kpt rather than in cli-utils.

ash2k commented 3 years ago

@mortent Thanks for clarifying this. Hm, that means ConfigMap as-is is not fit for this purpose and... cannot be used? So the default option in this library is something that people may/will have problems with. Maybe we can change how the names are encoded, like devising an escaping mechanism or just always turning data into base64 or maybe storing it as a JSON value in a single key in the ConfigMap? It's unfortunate that the default implementation has this shortcoming, I would expect it to "just work".

For kpt I agree a CRD makes sense and then it doesn't make sense to have ResourceGroup code in this repo. On the other hand it'd be awesome to standardize inventory objects so that different tools could interoperate and understand each other's inventory objects. From the user's perspective that would be very nice.

FYI @marshall007

bgrant0607 commented 3 years ago

@ash2k How are you using cli-utils? Do you have specific interoperability use cases in mind?

cli-utils provides some building blocks for tool builders, as well as reference client implementations for certain API features, but aspirations for interoperability have decreased over time.

Projects like Helm (https://helm.sh/docs/topics/advanced/#storage-backends) and Kapp (https://carvel.dev/kapp/docs/latest/) have chosen to avoid dependencies on CRDs that require installation by end users, interoperability of granular mechanisms across tools has not been prioritized highly by tool developers, there has been little adoption of https://github.com/kubernetes-sigs/application, and virtually all projects that do use CRDs define their own group types.

Similar to Helm storage backends, configurable or pluggable inventory resource types are probably necessary, for tools with different requirements, constraints, UX goals, etc. ConfigMap is a handy default because it's a built-in type. However, in addition to the other complications, simply intermingling with end user ConfigMaps creates challenges for UX, authorization, scaling, naming, watching, garbage collection, and so on.

While an escaping or encoding feature may be necessary for ConfigMap, it could impair the experience for CRDs, so it would need to be optional.

ash2k commented 3 years ago

@bgrant0607

How are you using cli-utils?

We are using cli-utils to perform apply and pruning in the GitOps feature of the GitLab Kubernetes Agent.

Do you have specific interoperability use cases in mind?

We support inventory object template being one of the objects in the repository. Would be nice if it was interoperable with kpt/other tools so that if the user needs to apply/etc objects from the repository manually, they could do it using the same object template. We would prefer not to come up with our own "standard", but use something the community standardized on. Looks like there is nothing at the moment.

I don't mind if it's a ConfigMap rather than a custom resource.

Regarding https://github.com/kubernetes-sigs/application - we are considering building a dashboard (or reusing an existing one if we find a suitable option) and thought it would make sense to support this specification. If there is no adoption, there is no point to support it indeed. Chicken and egg problem?

marshall007 commented 3 years ago

@bgrant0607 @ash2k I think it might be a mistake to conflate the discussion around how "inventory metadata" is persisted to the cluster (which may very well be tooling specific) and the mechanism by which a user signals to the tooling that the set of resources they are applying belong to the same "inventory".

This is different from something like the Application CRD because, just like the previous iteration using ConfigMap, the user does not expect that their on-disk representation of this resource is the same as what is ultimately applied to the cluster.

I think it makes sense for a user to be able to declare something like:

apiVersion: v1alpha1
kind: ResourceGroup
metadata:
  name: inventory-obj
  namespace: inventory-namespace
  labels:
    cli-utils.sigs.k8s.io/inventory-id: 87bb72f3-86cf-4096-a12c-67c8ac653679

... which arbitrary tooling can interface with as a signal of intent by the user regardless of how the tooling chooses to persist the inventory object to the cluster. That being said, I still agree with @ash2k that the SIG community should supply some standards for persisting the inventory object as well.

Another option would be to leverage the Application CRD, or a slightly re-imagined version of it that implements pruning logic on the client side instead of requiring a custom controller. If we are serious about it long term, the live apply commands are a great way to encourage adoption because they already require the user to specify some information about how to manage inventory.

I think users and tool developers would be much more incentivized to use an Application CRD that did not require an in-cluster component.

k8s-triage-robot commented 3 years ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot commented 3 years ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-triage-robot commented 2 years ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

k8s-ci-robot commented 2 years ago

@k8s-triage-robot: Closing this issue.

In response to [this](https://github.com/kubernetes-sigs/cli-utils/issues/364#issuecomment-1003638692): >The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. > >This bot triages issues and PRs according to the following rules: >- After 90d of inactivity, `lifecycle/stale` is applied >- After 30d of inactivity since `lifecycle/stale` was applied, `lifecycle/rotten` is applied >- After 30d of inactivity since `lifecycle/rotten` was applied, the issue is closed > >You can: >- Reopen this issue or PR with `/reopen` >- Mark this issue or PR as fresh with `/remove-lifecycle rotten` >- Offer to help out with [Issue Triage][1] > >Please send feedback to sig-contributor-experience at [kubernetes/community](https://github.com/kubernetes/community). > >/close > >[1]: https://www.kubernetes.dev/docs/guide/issue-triage/ Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
karlkfi commented 2 years ago

/remove-lifecycle rotten

k8s-triage-robot commented 2 years ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot commented 2 years ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

karlkfi commented 2 years ago

/remove-lifecycle rotten

Ugh. How do I turn off this useless bot?

karlkfi commented 2 years ago

/lifecycle frozen