research: provide sane default restricted pod security, limit access to mounting `/system`

[smira]

This is more of a research ticket to collect some data.

There are two problems we need to solve:

• Talos should provide sane default pod restriction policy (e.g. forbid running as root, privileged pods, etc.)
• We should limit Kuberntes workloads from mounting hostPaths which start with /system, because /system/state contains Talos config, /system/secrets contains PKI, and /system/run might contain unprotected file sockets

There might a single solution to both problems, or different solutions.

Today Talos deploys PSP which enables everything for every user in the system, so it's effectively no-op.

Potential solutions:

• PodSecurityPolicy, deprecated since 1.21, to be removed in 1.25 examples
• PodSecurity, new feature in alpha for 1.22, stable in 1.24 KEP
• Kyverno with policies as CRDs
• Kubewarden with policies as WASM programs
• Our own custom Admission webhook

Please also note that enabling write access to hostPath volumes means that eventually any path on the host filesystem can be mounted, so probably proper solution is simply to forbid hostPath volumes as baseline and allow it only for privileged users.

[andrewrynhard]

I would like to avoid any third party solutions if we can. That would be ideal IMHO.

[smira]

I would like to avoid any third party solutions if we can. That would be ideal IMHO.

we should probably discuss it more, but I wonder if we do PSP which forbids hostPath volumes as a baseline policy, should it be enough?

[andrewrynhard]

I would like to avoid any third party solutions if we can. That would be ideal IMHO.

we should probably discuss it more, but I wonder if we do PSP which forbids hostPath volumes as a baseline policy, should it be enough?

I feel like so. Our whole goal with Talos is to make the host irrelevant, so it makes sense to me.

[sanmai-NL]

@smira Given the current status on https://www.talos.dev/v1.6/kubernetes-guides/configuration/pod-security/, can this issue be closed?

[sanmai-NL]

@andrewrynhard @smira I think the main value that speaks from this issue, and that remains to be created, is a comprehensive documentation of Talos Linux's security model, applicable frameworks and the required parameters such as specific permissions. For example, document Talos Linux design wrt. Linux namespaces, networking, boot/disks and other dimensions. This would result in docs that list which filesystem paths are sensitive. When users then decide to reconfigure from the applied standards, they can be expected to take into account these paths and take appropriate measures themselves, with or without third party extensions to Kubernetes or Talos Linux.