jupyterhub / zero-to-jupyterhub-k8s

Helm Chart & Documentation for deploying JupyterHub on Kubernetes
https://zero-to-jupyterhub.readthedocs.io
Other
1.55k stars 797 forks source link

hook and continuous image pullers' DaemonSets: configuring k8s ServiceAccount - yes or no? #3545

Open consideRatio opened 1 day ago

consideRatio commented 1 day ago

Background

There are two sets of machinery to pull images to k8s nodes:

Both involves a k8s DaemonSet to schedule a pod on each node, and have that pod startup containers referencing images to "pre-pull", which makes the k8s node's pull the images.

However, the hook image puller DaemonSet resource is paired with the hook-image-awaiter k8s Job, that in turn have misc permissions to inspect k8s DaemonSets - and by doing that can know if the pulling has completed. When pulling is completed, the pre-upgrade helm hook can be considered completed, and the helm upgrade command can proceed. This is why the hook-image-awaiter k8s Job needs a k8s ServiceAccount for itself, so it can be granted permissions to ask the k8s api-server about DaemonSet resources.

Question

samyuh commented 1 day ago

I'm going to leave here the last comment that I left on the closed pull request :)

I really need to have every component in our deployment with a specific service account due to security compliance. We can just go with the option of setting the serviceAccountName with a value from the chart.

consideRatio commented 1 day ago

I've now updated the issue to raise two questions - is there a need to configure serviceAccountName or have the chart create a k8s ServiceAccount that is used, or is either fine?

Could you share some insights into why doing this could be relevant from a security perspective? I want to ensure complexity is only embraced for something a purpose that isn't addressed better in another way etc, for example by automountServiceAccountToken: false or similar.

samyuh commented 1 day ago

is there a need to configure serviceAccountName or have the chart create a k8s ServiceAccount that is used, or is either fine?

Either option is fine, as the end goal is to have a service account being assigned to the puller.

why doing this could be relevant from a security perspective?

We can't use default SA, since we need to explicitly set the permissions to be PCI-DSS compliant. Remember that the permissions can be either set in our side or on chart side.

consideRatio commented 1 day ago

Can you clarify even further, no permissions are granted these pods - but you need to explicitly grant them some additional permissions via a RoleBinding to a k8s ServiceAccount used by the pods?

I'm probing a lot now, because if this for example is for the sake of PSP deprecated in k8s 1.21 and removed in k8s 1.25 - its pointless i think.

samyuh commented 1 day ago

Yes, I'm going to gather more information about this and then I will reach back to you.

consideRatio commented 21 hours ago

I found this: https://cloud.google.com/kubernetes-engine/enterprise/policy-controller/docs/how-to/using-pci-dss-v4

pci-dss-v4.0-restrict-creation-with-default-serviceaccount Restricts the creation of resources using a default service account. Has no effect during audit.

I tried to figure out if all workloads must use one k8s serviceaccount at all, or if they could opt out entirely. It seems so far its either the default or another k8s serviceaccount, so complying with this requirement means to provide another i think.

samyuh commented 20 hours ago

Hey @consideRatio. This is the answer I got from our security team:

This is a requirement from a security framework that we are required to be in compliance with, CIS Benchmarks. More specifically the control in question is the 4.1.5 - Ensure that default service accounts are not actively used (from the EKS CIS Benchmark v1.5.0). A more practical explanation is that every workload has its specific "identity profile". This means that permissions for one workload are rarely the same across its entire lifecycle thus is always recommended for each workload to have a separate "identity". Not allowing the usage of the default service accounts helps in this topic.

We currently have a fork of this repository with the changes I suggested in the pull request that got closed but we would like to be using this public upstream release. Feel free to suggest other alternatives :)