Cluster is non-compliant with CIS benchmark because of /var/lib/etcd ownership

boomcubist commented 1 year ago

FEATURE REQUEST

Clusters deployed with kubeadm are non-compliant with section

1.1.12 Ensure that the etcd data directory ownership is set to etcd:etcd

because ownership of the directory is set to root:root.

Can it be that kubeadm creates an etcd user and group and assigns ownership /var/lib/etcd to etcd:etcd?

Versions

clientVersion:
  buildDate: "2023-03-15T13:40:17Z"
  compiler: gc
  gitCommit: 9e644106593f3f4aa98f8a84b23db5fa378900bd
  gitTreeState: clean
  gitVersion: v1.26.3
  goVersion: go1.19.7
  major: "1"
  minor: "26"
  platform: linux/amd64
kustomizeVersion: v4.5.7
serverVersion:
  buildDate: "2023-03-15T13:33:12Z"
  compiler: gc
  gitCommit: 9e644106593f3f4aa98f8a84b23db5fa378900bd
  gitTreeState: clean
  gitVersion: v1.26.3
  goVersion: go1.19.7
  major: "1"
  minor: "26"
  platform: linux/amd64

What happened?

/var/lib/etcd created with root:root ownership

What you expected to happen?

/var/lib/etcd created with etcd:etcd ownership

How to reproduce it (as minimally and precisely as possible)?

stat -c %U:%G /var/lib/etcd

neolit123 commented 1 year ago

Cluster is non-compliant with CIS benchmark because of /var/lib/etcd ownership

note that the CIS benchmark is not an approved benchmark by k8s core maintainers. this is something that a lot of users don't understand. the benchmark reports are quite opinionated and sometimes a false-positive.

1.1.12 Ensure that the etcd data directory ownership is set to etcd:etcd

what is the argument for that? is it because the etcd pod should not run as root and an etcd:etcd user must own the etcd lib dir?

we actually wanted to add a feature to change the root ownership in kubeadm: https://github.com/kubernetes/enhancements/tree/master/keps/sig-cluster-lifecycle/kubeadm/2568-kubeadm-non-root-control-plane

but please do not use that feature gate as we might remove it, in favor of user namespaces. https://github.com/kubernetes/enhancements/blob/16d5ddfe78b0f561df278ec71231beb7774f581d/keps/sig-node/127-user-namespaces/README.md?plain=1#L319 they are still WIP.

can you use this kubeadm flag?

--rootfs string
--
  | [EXPERIMENTAL] The path to the 'real' host root filesystem.

or symlinks to get around the CIS benchmark issue report?

boomcubist commented 1 year ago

The rationale for the control is

etcd is a highly-available key-value store used by Kubernetes deployments for persistent storage of all of its REST API objects. This data directory should be protected from any unauthorized reads or writes. It should be owned by etcd:etcd.

It looks a lot running the control-plane as non-root would solve this problem.

neolit123 commented 1 year ago

running the kubeadm control plane pod as non root makes sense from a security stand point and has been discussed / planned already. but our feature that we added RootlessControlPlane is alpha and will not be updated to beta because of the user namespaces plan: https://github.com/kubernetes/enhancements/blob/16d5ddfe78b0f561df278ec71231beb7774f581d/keps/sig-node/127-user-namespaces/README.md?plain=1#L319

once the user namespaces feature graduates to beta we can intergrate it in kubeadm. not much we can do before that.

you can use RootlessControlPlane if you'd like, but as noted it will be removed at some point. ...or maybe we can think of a smoother transition and wrap user namespaces with our existing RootlessControlPlane feature gate. TBD...

kubernetes / kubeadm