Closed T-Kukawka closed 7 months ago
I checked the feature gates mentioned from above
k8s api feature gates
TTLAfterFinished=true - GA and active since 1.23, we do not need to add this JobTrackingWithFinalizers=true - Beta and active by default since 1.25, we do not need to add this EphemeralContainers=true - Beta and active by default 1.23, we do not need to add this
k8s Controller Manager
TTLAfterFinished=true - see above no action needed CSIMigrationAWS=true - Beta and active since 1.23, we do not need to add this JobTrackingWithFinalizers=true - see above
Regarding Feature Gates in general, I think it would make sense having a feature gate configuration for each component but only BETA:
.Values.internal.advancedConfiguration.controlPlane.kubelet
and .Values.internal.advancedConfiguration.workers.kubelet
.Values.internal.advancedConfiguration.controlPlane.apiServer
.Values.internal.advancedConfiguration.controlPlane.controlerManager
.Values.internal.advancedConfiguration.controlPlane.kubeScheduler
For documentation purposes, i would ask to add each topic as a separate entry in directory such as: https://github.com/giantswarm/capi-migration-cli/pull/13
I read https://github.com/giantswarm/giantswarm/issues/28558 and compared it to what's listed here and what @njuettner stated above.
@T-Kukawka: If I'm getting everything right, the only three features we need to implement to get this story done are the following:
PodNodeSelector
Admission ControllercontainerLogMaxSize
Is this correct?
correct @Gacko 👍
We already have a quite extensive audit policy in place for CAPI: https://github.com/giantswarm/cluster/blob/main/helm/cluster/files/etc/kubernetes/policies/audit-policy.yaml
well, customers like to set their own as on Vintage, u can refer to the discovery ticket @Gacko
Ok, fine. So I think we just need the possibility to add additional rules to the existing policy.
yeah, exactly 👍
@Gacko can u provide documentation for those features? or references so we can document as well as integrate that in the migration CLI?
Sure, I'll put something in the README.md
of this repository. IIRC the old way of using the k8s-initiator-app
was a bit tricky as you needed to change stuff in the code to customize it, but now it's more about setting the right values, that's it.
@Gacko i have added this for the node-tainting - i believe we should do the same for your case: https://github.com/giantswarm/capi-migration-cli/blob/main/k8s-initiator-features/node-tainting.md
We can have a call if u like
Currently Vintage is a heavy user of k8s initiator app which allows customers to customize the settings directly on the nodes file systems/configuration. Following the investigation of the currently used features: https://github.com/giantswarm/giantswarm/issues/28558, we have to make the adjustments to CAPA such that the migration of given features is possible and there are 'features' enabled to migrate to.
Feature Gates
As a customer i would like to be able to enable the features gates available on the k8s. Currently on Vintage this part is done via the k8s-initiator-app.
There are several groups of feature gates used by customers:
The k8s api feature gates such as:
TTLAfterFinished=true
JobTrackingWithFinalizers=true
EphemeralContainers=true
k8s Controller Manager:
TTLAfterFinished=true
CSIMigrationAWS=true
JobTrackingWithFinalizers=true
Admission controller:
k8s API configuration
Additionally via k8s-initiator-app it is available to customers to edit the e.g.
audit-policy
for k8s api such as: https://github.com/giantswarm/k8s-initiator-app/blob/main/examples/values_audit_policy.yamlThis feature is extensively used by one customer, where several policies are applied per cluster.
kubelet max log size
This feature is also used by one customer: