Allow "safe" operations of CSI plugins

hashicorp / nomad

Nomad is an easy-to-use, flexible, and performant workload orchestrator that can deploy a mix of microservice, batch, containerized, and non-containerized applications. Nomad is easy to operate and scale and has native Consul and Vault integrations.

https://www.nomadproject.io/

Other

14.91k stars 1.95k forks source link

Allow "safe" operations of CSI plugins #9258

Open apollo13 opened 4 years ago

apollo13 commented 4 years ago

Currently CSI plugins are usually required to run as privileged to be able to mount stuff on the host. This (sadly) also requires one to allow "privileged" for all other containers on the node.

If it were possible that CSI plugins would implicitly add privileged=true (I think this should be possible; after all it also adds a bindmount to /csi which also would be denied by the default docker plugin configuration), then one would not have to allow privileged=true for all other jobs.

Now to prevent submission of all jobs as csi plugins one can use the namespace capability from https://learn.hashicorp.com/tutorials/nomad/access-control-policies?in=nomad/access-control#csi-register-plugin to limit this to certain users.

What do you think? All in all I think this would greatly improve the security of nomad when CSI plugins are in use.

tgross commented 4 years ago

Hi @apollo13! This is definitely something we considered. There's an issue https://github.com/hashicorp/nomad/issues/6554 that covers it in some more detail, but I'll keep this issue open as a specific use case of #6554.

the-maldridge commented 3 years ago

I'd just like to chip in on this as well. While Nomad is a trusted operator model and I generally trust the people I let have access to it, that trust doesn't come in a single level. I might trust people to launch arbitrary docker services, but not to launch things that need to run with higher permissions granted.

tgross commented 3 years ago

I'm going to clarify this issue a bit. There are two pieces to this idea:

Making plugin jobs automatically "privileged": we'd have to do in the Docker task driver, because "privileged" is not a Nomad concept but a Docker one. Note that there's no security benefit to allowing a user to launch plugins without also implying they have the ability to launch privileged jobs. A user who wanted to escalate their access to the client host could deploy an arbitrary workload with a csi_plugin block and it would be identical in all ways to a privileged task.
Having an ACL that denies users the ability to launch tasks as privileged (perhaps except for plugin jobs): this is where the real security benefit comes in, because it's much finer-grained than the host-level configuration.

ryan-omni3 commented 3 years ago

Being able to set an ACL for the privileged flag would be ideal. Maybe in combination with an "allow" list of containers listed in the docker driver stanza that are permitted to run privileged. Regardless, it is on the admin to monitor what tasks are being ran by their users, and inappropriate use handled accordingly.

suikast42 commented 1 year ago

Any progress on this ?

tgross commented 1 year ago

It's not on the nearterm roadmap. Generally speaking if we have updates on what we're doing, you'll see them show up either on the issue or on https://github.com/orgs/hashicorp/projects/202/views/1