search

hashicorp / nomad

Nomad is an easy-to-use, flexible, and performant workload orchestrator that can deploy a mix of microservice, batch, containerized, and non-containerized applications. Nomad is easy to operate and scale and has native Consul and Vault integrations.
https://www.nomadproject.io/
Other
14.76k stars 1.94k forks source link

[feature] FS Isolation with UnionFS #1546

Closed parasyte closed 2 years ago

parasyte commented 8 years ago

Tools like Docker use UnionFS to provide FS resource isolation without the overhead of chroot. It should make for a powerful combination; chroot for control over what appears in the FS, and UnionFS for its copy-on-write semantics (many performance benefits).

A one-time step during startup can create a basic root file systems for all jobs to share, and the executors use it as a read-only UnionFS branch with a job-specific read-write branch to allow writes. And wrap it all up with chroot on the new mount point.

schmichael commented 7 years ago

Related to #2355 although we'll probably want a native Linux solution like btfrs, aufs, overlayfs, etc.

schmichael commented 2 years ago

While this is still a good idea, at this point I think we're more likely to create a new driver (exec2? maybe just something more vanilla containerd?) that does this.

At this point changing the exec driver in place is pretty risky, so I'm going to close this. I can't imagine anyone implementing a new driver wouldn't choose one of the modern chroot solutions we mention here.

github-actions[bot] commented 1 year ago

I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

schmichael commented 2 months ago

While this is still a good idea, at this point I think we're more likely to create a new driver (exec2? maybe just something more vanilla containerd?) that does this.

A few years later and making a new exec2 driver is exactly what we did: https://github.com/hashicorp/nomad-driver-exec2

It uses Landlock LSM instead of an overlay filesystem, so it's a slightly different solution than what is proposed here. While we may still expand our use of overlay filesystems in the future, I think Landlock is compelling alternative and arguably much lighter weight.

(This is a good example of why we stopped using stalebot: we do get to these old issue eventually!)