Allow configuration of kubelet feature-gates

[nickperry]

Is this a FEATURE REQUEST? YES

• why: We need to be able to enable feature gates like DynamicKubeletConfig for the kubelet.
• what: The module currently does not currently seem to support configuration of feature-gates, although it is partially implemented. See https://github.com/puppetlabs/puppetlabs-kubernetes/blob/7fb46f2a77bddd3ac294bb5fd22fc62fcc16381d/lib/puppet/parser/functions/kubeadm_init_flags.rb#L17 and https://github.com/puppetlabs/puppetlabs-kubernetes/blob/7fb46f2a77bddd3ac294bb5fd22fc62fcc16381d/manifests/kubeadm_init.pp#L12

Versions:

3.1

$ puppet --version
4.7.0
$ kubectl version
v1.11.3
$ docker version OR crictl version
17.03.0-ce
$ facter os
{
  architecture => "x86_64",
  distro => {
    codename => "n/a",
    description => "Oracle Linux Server release 7.5",
    id => "OracleServer",
    release => {
      full => "7.5",
      major => "7",
      minor => "5"
    },
    specification => ":core-4.1-amd64:core-4.1-noarch"
  },
  family => "RedHat",
  hardware => "x86_64",
  name => "OracleLinux",
  release => {
    full => "7.5",
    major => "7",
    minor => "5"
  },
  selinux => {
    enabled => false
  }
}

[nickperry]

Actually

Exec[kubeadm init]/returns: can not mix '--config' with arguments [feature-gates]

So I can enable feature-gates via the config using kubeadm_extra_config. However, this only works for the controllers, not worker nodes at this time. This is because the module does not write a config file and pass it to kubeadm for join - only for init.

[nickperry]

Fundamentally, the higher level problem I'm tying myself in knots trying to fix is how to use this module to define a custom infra / pause container for use in our air-gapped environments. This was easy to achieve for controllers, through the addition of $kubelet_extra_arguments. However, the fact that join doesn't use a config file whereas init does in this module, is making this tricky for worker nodes.

I have realised I can work around this by managing /etc/sysconfig/kubelet from my profile manifest which calls the kubenernetes module.