kptdev / kpt

Automate Kubernetes Configuration Editing
https://kpt.dev
Apache License 2.0
1.7k stars 227 forks source link

ConfigMap generation #3119

Open bgrant0607 opened 2 years ago

bgrant0607 commented 2 years ago

ConfigMap generation was one of the first generators built for kustomize. It's pretty common: env files, app config files that need to be injected via volume mount.

We'll need to figure out how to support this.

It's a special case of generator (#2528).

bgrant0607 commented 2 years ago

Also, kustomize appends content-based hashes to the suffixes of ConfigMap names in order to facilitate rollouts: https://github.com/kubernetes/kubernetes/issues/22368

However, there's no mechanism in Kubernetes to garbage collect them. https://groups.google.com/g/kubernetes-sig-apps/c/d_zV4esoNF8 https://github.com/kubernetes/community/pull/1163/files

Previous versions should not be pruned immediately because they are used by previous ReplicaSets during rolling updates, so that requires special pruning logic to not delete them too soon, and perhaps to delete them when they are no longer used.

We may want to look at approaches that handle this at run time. https://github.com/kubernetes/enhancements/pull/948 https://github.com/mattmoor/boo-maps https://github.com/gardener/gardener-resource-manager/issues/104 https://www.tigera.io/blog/solving-kubernetes-configuration-woes-with-a-custom-controller/ https://github.com/gopaddle-io/configurator

Without changes to the core workload controllers, though, a runtime mechanism that changed the pod template would conflict with GitOps controllers, but there's no other way to make pod template revisions point at specific ConfigMap revisions. Changing an underlying ReplicaSet out of band would fight with the Deployment controller.

This could be done client-side, as kustomize does. Garbage collection could also be done client-side, by keeping the same number of ConfigMap revisions as the history setting on the workload controllers that consume them. That would avoid a change to the pruning mechanism, also.

bgrant0607 commented 2 years ago

Helm support for triggering rollouts: https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments

yuwenma commented 2 years ago

ConfigSync uses an annotation (ConfigMap hash) to detect ConfigMap change and re-apply the referrer resources

bgrant0607 commented 2 years ago

kubectl support: https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/

kustomize: https://github.com/kubernetes-sigs/kustomize/blob/master/examples/configGeneration.md https://kubectl.docs.kubernetes.io/references/kustomize/builtins/#_configmapgenerator_ https://kubernetes.io/docs/tasks/manage-kubernetes-objects/kustomization/#configmapgenerator