search

hashicorp / nomad

Nomad is an easy-to-use, flexible, and performant workload orchestrator that can deploy a mix of microservice, batch, containerized, and non-containerized applications. Nomad is easy to operate and scale and has native Consul and Vault integrations.
https://www.nomadproject.io/
Other
14.92k stars 1.95k forks source link

Recognize nomadVar alloc index under job #22235

Open albttx opened 5 months ago

albttx commented 5 months ago

Proposal

Today, it's possible to store variable in Nomad and we can ACL's variable based on the path

{{ with nomadVar "nomad/jobs/group_name/task_name" }}

My proposal is to add a 4th layer of the alloc_index

{{ with nomadVar "nomad/jobs/job_name/group_name/task_name/1" }}
{{ with nomadVar "nomad/jobs/job_name/group_name/task_name/2" }}
{{ with nomadVar "nomad/jobs/job_name/group_name/task_name/${NOMAD_ALLOC_INDEX}" }}

Use-cases

Having a group with multiple tasks don't means to have the same variables.

Attempted Solutions

The only solution to keep ACL's working is to have the index in the variable names.

I can trick using something like

{{- define "custom" }}MY_VALUE_{{ NOMAD_ALLOC_INDEX }}{{ end }}
{{ executeTemplate "custom" }}: {{ env (executeTemplate "custom") }}

Because if you do

nomad/jobs/job_name/group_name/task_name/1

You lost the the ACL permission on the task name

tgross commented 5 months ago

Hi @albttx! Yeah there's definitely no way to hack this in yourself with an ACL policy right now, because Nomad's ACL policies don't accept any kind of interpolation outside of wildcards.

This is at least technically possible... we'd need to add a nomad_alloc_index field to every Workload Identity Nomad signs. That'd increase the WI JWT size by 32 bytes or so, which isn't too bad. But that does add up and we should be thoughtful about the value of adding to every JWT because we're stuck with those decisions for a long time. I'll mark this for further discussion and roadmapping.