search

kubedl-io / kubedl

Run your deep learning workloads on Kubernetes more easily and efficiently.
https://kubedl.io/
Apache License 2.0
500 stars 78 forks source link

[BUG]A potential risk of kubedl which can be leveraged to make a cluster-level privilege escalation #316

Open younaman opened 2 months ago

younaman commented 2 months ago

Dear kubedl maintainers: I am Nanzi Yang, a PostDoc of University of Minnesota. I find a potential risk of kubedel which can be leveraged to make a cluster-level privilege escalation.

Detailed analysis: The kubedl has a deployment called default-kubedl, which is bound with a ClusterRole called default-kubedl-role. This ClusterRole has create verb of pods resources, and has create/patch/update verbs of deployment resource(https://github.com/kubedl-io/kubedl/blob/master/helm/kubedl/templates/role.yaml#L91). If a malicious user can access the worker node which has this deployment, he/she can abuse these permissions to create or modifying a set of Pods and specify a high-privilege service account for these pods/deployments. After that, he/she can abuse the high-privilege SA token of the created pods to do whatever he/she likes to the whole cluster, resulting in a cluster-level privilege escalation.

Mitigation discussion:

  1. The maintainers can try to use PodSecurityPolicies, Pod Security Standards, or Admission Controllers to restrict which ServiceAccounts can be specified when creating Pods. This can prevent users from assigning high-privilege ServiceAccounts like cluster-admin to their Pods.
  2. You can try to review the source code and remove these excessive permissions if it's not needed by this deployment.
  3. You can also try to use the Kubernetes namespace and RoleBinding instead of ClusterRoleBinding to eliminate the cluster-level effects.

A few questions:

  1. Is it a real issue of kubedl?
  2. If it's a real issue, can kubedl fix these problems following my suggestions?
  3. If it's a real issue, can kubedel give me a CVE number for awarding my efforts?

By the way, I tried to send a message to kubedlalibaba@gmail.com to report this issue. However, it seems that my mail-box can not find this email address. After that, I sent message to kubedl owners privately, and qiukai.cqk@alibaba-inc.com (Qiukai, one of the owners) asked me to post a github issue publicly. So I open this issue to make a further discussion:) Looking forward to your reply Regards, Nanzi Yang

SimonCqk commented 2 months ago

@younaman hi naman, thank you for bringing this to our attention. We understand your concerns, and indeed, RBAC authorization is a common practice for managing Kubernetes components, and these permissions are essential. That said, we will review the permissions included in our ClusterRole based on your feedback.

sparkEchooo commented 1 month ago

@SimonCqk Knock knock! It has been two weeks. Are there any updates on your review? Looking forward to your reply. Looking forward to your reply. Regards, Nanzi Yang