search

awslabs / amazon-eks-ami

Packer configuration for building a custom EKS AMI
https://awslabs.github.io/amazon-eks-ami/
MIT No Attribution
2.44k stars 1.15k forks source link

bootstrap.sh does'nt support kubelet config file #1289

Closed langavriel closed 1 year ago

langavriel commented 1 year ago

What would you like to be added: The bootstrap.sh file only supports the --kubelet-extra-args flag that is deprecated on eks 1.26, and does not support the kubelet --config flag that allows to pass kubelet configuration files to kubelet.

Why is this needed: As of eks 1.26, passing arguments to the kubelet is deprecated and is only supported via a config file, as stated here: https://kubernetes.io/blog/2022/11/18/upcoming-changes-in-kubernetes-1-26/#removal-of-dynamic-kubelet-configuration

stevehipwell commented 1 year ago

@langavriel the deprecation you've linked to isn't related to the --kubelet-extra-args flag as the flag is part of bootstrap.sh and not Kubernetes. The --kubelet-extra-args bootstrap flag is used to define KUBELET_EXTRA_ARGS which is a convention to allow for the static customisation of kubelet.

langavriel commented 1 year ago

Hi @stevehipwell , Is this env var supported on eks 1.26? I tested this flag on eks 1.26 and it did not apply the configuration (and it did apply the changes on eks 1.25), so i dont think this is supported on eks 1.26+. Anyways, there is no flag that supports passing a kubeket config file which could be very helpful.

stevehipwell commented 1 year ago

@langavriel the answer to your questions can be found in the kubelet systemd unit; the config is defined as part of the AMI (/etc/kubernetes/kubelet/kubelet-config.json) and the KUBELET_EXTRA_ARGS are supported (they're added to the unit spec in bootstrap.sh).

Please note that if you're using MNGs then you'll need to work around the limitations of cloud-init (e.g. patching bootstrap.sh for persistence).

cartermckinnon commented 1 year ago

@langavriel sorry for the late reply, I've been out of the office. This bootstrap flag is intended to pass arbitrary args to the kubelet command line. You can use it to define an alternative kubelet config file, and it should "just work" -- kubelet should use the last value for a given flag (if only one value is allowed). I'd recommend trying that instead of modifying the existing kubelet config file in-place, because the bootstrap script may overwrite your changes.