search

hashicorp / nomad

Nomad is an easy-to-use, flexible, and performant workload orchestrator that can deploy a mix of microservice, batch, containerized, and non-containerized applications. Nomad is easy to operate and scale and has native Consul and Vault integrations.
https://www.nomadproject.io/
Other
14.83k stars 1.95k forks source link

Root signing key rotation and prepublishing #19669

Closed schmichael closed 2 months ago

schmichael commented 8 months ago

Nomad 1.7 uses a root encryption key to encrypt Variables at rest and a root signing key to sign Workload Identities.

These root keys should be rotated automatically using the following logic:

  1. New root key should be generated at root_key_rotation_threshold / 2 and the public signing key published before use in the JWKS endpoint.
  2. At root_key_rotation_threshold the prepublished key will be made active and the old active key will be made inactive
  3. After root_key_rotation_threshold + root_key_gc_threshold after the old key was marked inactive, it should be garbage collected.
    • Update root_key_* docs to reflect that keys are not gc'd until rotation_threshold + gc_threshold are reached to avoid invalidating otherwise valid JWTs in use.
  4. Jobspecs with an identity.tll > root_key_rotation_threshold should receive a Warning on submit.

Prior Art

https://github.com/hashicorp/vault/pull/12414

tgross commented 8 months ago

What you're envisioning here should cover https://github.com/hashicorp/nomad/issues/19367 and https://github.com/hashicorp/nomad/issues/19368. I'm going to unassign myself from those and if you want, we can either close them out now or you can close them out with this issue.

schmichael commented 8 months ago

Thanks for linking things together Tim.

we can either close them out now or you can close them out with this issue.

I'm going to leave them open until this ships to ensure everything is buttoned up appropriately.

tgross commented 2 months ago

Implemented in #23577 and will ship in the next regular release of Nomad 1.8.x, with backports to Nomad 1.7.x/1.6.x Enterprise.