puppetlabs / puppetlabs-kubernetes

This module install and configures a Kubernetes cluster
Apache License 2.0
92 stars 135 forks source link

Make cgroup_driver default to systemd #631

Closed r-tierney closed 1 year ago

r-tierney commented 1 year ago

As per prior discussion within this PR I've made systemd the default cgroup_driver for the next major release or when ready. https://github.com/puppetlabs/puppetlabs-kubernetes/pull/625 https://github.com/puppetlabs/puppetlabs-kubernetes/pull/625#issuecomment-1551496610

TLDR; The default cgroup_driver for kubelet is systemd, however the default within this module for containerd is set to cgroupfs which causes a conflict between containerd & kubelet leading to kubelet crashlooping. This PR is to set the default cgroup_driver for containerd to systemd as recommended by the kubernetes team Kubernetes Docs for this Tested on Debian bullseye & bookworm.

There are still a few files which have this set to cgroupfs as seen here:

ryant@Ryans-MacBook-Air:~/puppetlabs-kubernetes on  systemd_cgroup_driver_default took 2s ❯ grep -r cgroupfs
spec/classes/packages_spec.rb:      'docker_cgroup_driver' => 'cgroupfs',
spec/classes/packages_spec.rb:    it { is_expected.to contain_file('/etc/docker/daemon.json').with_content(%r{\s*"native.cgroupdriver=cgroupfs"\s*}) }
spec/spec_helper_acceptance_local.rb:      cgroup_driver => 'cgroupfs',
spec/spec_helper_acceptance_local.rb:      cgroup_driver => 'cgroupfs',
spec/spec_helper_acceptance_local.rb:      cgroup_driver => 'cgroupfs',
spec/acceptance/kubernetes_spec.rb:              cgroup_driver => 'cgroupfs',
ryant@Ryans-MacBook-Air:~/puppetlabs-kubernetes on  systemd_cgroup_driver_default ❯
puppet-community-rangefinder[bot] commented 1 year ago

kubernetes::config::kubeadm is a class

that may have no external impact to Forge modules.

kubernetes::config::worker is a class

that may have no external impact to Forge modules.

kubernetes is a class

Breaking changes to this file MAY impact these 5 modules (near match): * [jetstack-tarmak](https://github.com/jetstack/tarmak/tree/master/puppet/modules/tarmak) * [jetstack-fluent_bit](https://github.com/jetstack/tarmak/tree/master/puppet/modules/fluent_bit) * [jetstack-prometheus](https://github.com/jetstack/tarmak/tree/master/puppet/modules/prometheus) * [jetstack-calico](https://github.com/jetstack/tarmak/tree/master/puppet/modules/calico) * [jetstack-kubernetes_addons](https://github.com/jetstack/tarmak/tree/master/puppet/modules/kubernetes_addons)

kubernetes::packages is a class

that may have no external impact to Forge modules.

This module is declared in 0 of 580 indexed public Puppetfiles.


These results were generated with Rangefinder, a tool that helps predict the downstream impact of breaking changes to elements used in Puppet modules. You can run this on the command line to get a full report.

Exact matches are those that we can positively identify via namespace and the declaring modules' metadata. Non-namespaced items, such as Puppet 3.x functions, will always be reported as near matches only.

jordanbreen28 commented 1 year ago

@r-tierney that was fast! I think if we are going to default to systemd we should update the test cases you linked in your original comment to use this driver too. Means we can spot any potential issues using the CI.

r-tierney commented 1 year ago

@r-tierney that was fast! I think if we are going to default to systemd we should update the test cases you linked in your original comment to use this driver too. Means we can spot any potential issues using the CI.

Sure thing, thats updated in my latest commit

Here is whats left as i haven't dug into the docker_cgroup_driver so left that unchanged:

ryant@Ryans-MacBook-Air:~/puppetlabs-kubernetes on  systemd_cgroup_driver_default took 3s ❯ grep -r cgroupfs
spec/classes/packages_spec.rb:      'docker_cgroup_driver' => 'cgroupfs',
spec/classes/packages_spec.rb:    it { is_expected.to contain_file('/etc/docker/daemon.json').with_content(%r{\s*"native.cgroupdriver=cgroupfs"\s*}) }
ryant@Ryans-MacBook-Air:~/puppetlabs-kubernetes on  systemd_cgroup_driver_default ❯
david22swan commented 1 year ago

Will merge in and release alongside: https://github.com/puppetlabs/puppetlabs-kubernetes/pull/633