Add /etc/kubernetes to policy/policies

First, a HUGE disclaimer: I have very little knowledge of SELinux and am usually struggling as a normal user always suggesting disabling SELinux. 😉

First my use case: We want to run https://secrets-store-csi-driver.sigs.k8s.io/ on a RKE2 cluster. Because our target production Kubernetes distro is Openshift, where SELinux is enabled by default, we have enabled SELinux in the RKE2 cluster - to make the environments more similar. While the CSI driver seems to work as it should, I am unable to successfully install any Secrets Store CSI Driver providers. We have attempted with the Vault provider, but I'll guess we would face the same issue with any provider. The problem seems to be that SELinux blocks the Vault CSI pod(s) from creating a socket in the providers directory to interact with the Secrets Store CSI Driver. This is the error message (from within the Vault CSI pod):

2023-08-17T09:18:06.799Z [INFO]  Creating new gRPC server
2023-08-17T11:18:06.799638382+02:00 2023-08-17T09:18:06.799Z [INFO]  Opening unix socket: endpoint=/provider/vault.sock
2023-08-17T11:18:06.800496999+02:00 2023-08-17T09:18:06.800Z [ERROR] Error running provider: err="failed to listen on unix socket at /provider/vault.sock: listen unix /provider/vault.sock: bind: permission denied"

The /provider directory is mounted into the container from the /etc/kubernetes/secrets-store-csi-providers host path.

Comparison to Openshift

We usually experience more issues on Openshift (secure by default), but in this particular case this actually works. And we think this is because Openshift adds special SELinux handling of /etc/kubernetes.

Openshift:

sh-5.1# ls -laZ /etc/kubernetes/
total 52
drwxr-xr-x.  7 root root system_u:object_r:kubernetes_file_t:s0   212 Aug 16 12:42 .
drwxr-xr-x. 96 root root system_u:object_r:etc_t:s0              8192 Aug 16 12:41 ..
-rw-r--r--.  1 root root system_u:object_r:kubernetes_file_t:s0  1123 Aug 16 12:39 ca.crt
-rw-r--r--.  1 root root system_u:object_r:kubernetes_file_t:s0     0 Aug 16 12:39 cloud.conf
drwxr-xr-x.  3 root root system_u:object_r:kubernetes_file_t:s0    19 Aug 16 12:40 cni
-rw-r--r--.  1 root root system_u:object_r:kubernetes_file_t:s0 12649 Aug 16 12:39 kubeconfig
-rw-r--r--.  1 root root system_u:object_r:kubernetes_file_t:s0 16194 Aug 16 12:42 kubelet-ca.crt
drwxr-xr-x.  3 root root system_u:object_r:kubernetes_file_t:s0    20 Aug 16 12:40 kubelet-plugins
-rw-r--r--.  1 root root system_u:object_r:kubernetes_file_t:s0  2001 Aug 16 12:39 kubelet.conf
drwxr-xr-x.  2 root root system_u:object_r:kubernetes_file_t:s0     6 Aug 16 12:40 manifests
drwxr-xr-x.  2 root root system_u:object_r:kubernetes_file_t:s0    24 Aug 16 12:41 secrets-store-csi-providers
drwxr-xr-x.  3 root root system_u:object_r:kubernetes_file_t:s0    24 Aug 16 12:40 static-pod-resources

RKE2:

[root@<redacted> /]# ls -laZ /etc/kubernetes/
total 12
drwxr-xr-x.   3 root root system_u:object_r:etc_t:s0   41 Aug 17 11:00 .
drwxr-xr-x. 106 root root system_u:object_r:etc_t:s0 8192 Aug 17 11:00 ..
drwxr-xr-x.   2 root root system_u:object_r:etc_t:s0    6 Aug 17 11:00 secrets-store-csi-providers

Suggestion

It seems to me that /etc/kubernetes is a common location for Kubernetes-stuff, ref. this Google search, so I would suggest adding rules for this directory in this project. Similar to what Openshift do in their SELinux-profile - which will allow pods with host path access to work on contents.

Available workaround

For now, I'll probably use https://github.com/kubernetes-sigs/security-profiles-operator to apply an SELinux policy to the RKE2 nodes. This is adding complexity to our setup and is something I would like to avoid.

rancher / rke2-selinux