search

hashicorp / nomad

Nomad is an easy-to-use, flexible, and performant workload orchestrator that can deploy a mix of microservice, batch, containerized, and non-containerized applications. Nomad is easy to operate and scale and has native Consul and Vault integrations.
https://www.nomadproject.io/
Other
14.83k stars 1.95k forks source link

Respect the case type of capabilities in cap_add #12354

Open cvandal opened 2 years ago

cvandal commented 2 years ago

Proposal

When a capability is added to cap_add, the case type should be respected and passed through to the driver as is.

For example, setting cap_add = ["NET_RAW"] should result in "CapAdd": ["NET_RAW"] when running docker inspect <container_id>, and setting cap_add = ["net_raw"] should result in "CapAdd": ["net_raw"] when running docker inspect <container_id>.

Use-cases

The SumoLogic collector for Docker looks for the NET_RAW capability, specifically in uppercase, and because Nomad transforms it to lowercase it fails to find it and throws the following error:

Cannot deserialize value of type `com.github.dockerjava.api.model.Capability` from String "net_raw": not one of the values accepted for Enum class: [ALL, SYS_BOOT, DAC_OVERRIDE, NET_RAW, BLOCK_SUSPEND, FOWNER, IPC_LOCK, IPC_OWNER, SYS_PACCT, NET_BIND_SERVICE, WAKE_ALARM, FSETID, DAC_READ_SEARCH, SYS_CHROOT, SYS_RAWIO, SYS_ADMIN, KILL, MAC_ADMIN, SYS_RESOURCE, CHOWN, SETPCAP, SYS_PTRACE, NET_ADMIN, SETFCAP, SYS_NICE, LINUX_IMMUTABLE, AUDIT_CONTROL, LEASE, AUDIT_WRITE, SYS_MODULE, MKNOD, SYSLOG, MAC_OVERRIDE, SYS_TIME, SETGID, SETUID, SYS_TTY_CONFIG, NET_BROADCAST]

Attempted Solutions

I have tried setting cap_add = ["NET_RAW"] in my job HCL template and allow_caps = [..., "NET_RAW"] in my client plugin stanza.

tgross commented 2 years ago

Hi @cvandal! This actually startled me at first because I was worried that canonicalizing the capability name incorrectly would remove the cap. But I've checked that's working as expected.

I'm pretty sure the reason that we canonicalize the names rather than taking them raw is so that we can de-duplicate the capabilities when we compare them against the driver's own set of allowed caps. (I do kind of wish that in retrospect we'd canonicalized them to match the Linux constants names exactly.) In any case I think following Docker's behavior makes sense for the Docker driver, at least. Will mark this for fixing.

shoenig commented 2 years ago

Hey @cvandal if you're interested in working up a fix, I think the way to go about it would be to have this Set[0] map from the normalized name to the user-given name, rather than nothing. Then, fix/copy this Slice[1] function to return the original values.

[0] https://github.com/hashicorp/nomad/blob/v1.2.6/drivers/shared/capabilities/set.go#L22 [1] https://github.com/hashicorp/nomad/blob/v1.2.6/drivers/shared/capabilities/set.go#L112