Key rotation

[mtcolman]

Hello, does tekton chains support rotation of the keys used to sign artifacts, pipelineRuns, taskRuns etc?

The scenario we have is where we are using chains with hashicorp vault and ideally we'd like to sign with keys that are rotated either for every pipelineRun, or once a day, week etc.

We have discussed looking at a mechanism to delete the key to force generation of a new one, however we have mutliple pipelines running in parallel and see we can't be certain that a key is "finished with" unless we halt all running pipelines to perform the rotation ourselves.

I can't find any documentation on this, so thought I'd ask here.

Thanks,

Matt

[lcarva]

@mtcolman, unfortunately Chains doesn't have any controls for key rotation. It does load the signing key each time it needs it, so it should pick up updates without having to restart the controller.

This would be challenging to get it right because Chains processes both TaskRuns and PipelineRuns. It is possible that an image and TaskRun SLSA Provenance are signed with one key, while the PipelineRun SLSA Provenance is signed with another.

[wlynch]

I suspect handling key phasing on the verification side might be easier - I don't think we'd want to stop reconciliation on the Chains to phase out a key (since pipelines could take O(hours) to complete), so there's always going to be some amount of overlap. Even if we pinned keys for all sub-resources in a pipeline, because pipelinerun times can vary so drastically it would be difficult to truly know at what point in time a key is finally phased out. You might be able to guesstimate a safe deletion time by using time when Chains config was rotated + max timeout for your pipelines + some wiggle room, but I suspect you want to delete the private keys ASAP. 🙃

You might be interested in the Sigstore Keyless integration, which will generate fresh keys client side per request backed by a code-signing cert from the Sigstore Fulcio CA, and never persist them.

[mtcolman]

@lcarva could you please clarify "It does load the signing key each time it needs it, so it should pick up updates without having to restart the controller." - so if we rotate the key ourselves in Vault, chains will pick up the new key when it starts a new pipelineRun?

[lcarva]

so if we rotate the key ourselves in Vault, chains will pick up the new key when it starts a new pipelineRun?

I believe so. To be clear, Chains only processes a TaskRun when it is completed, and only processes a PipelineRun when it is completed and it is done processing all the included TaskRuns. In other words, if you rotate the key after the PipelineRun starts, but before it ends, Chains should use the new key when processing that PipelineRun.

[tekton-robot]

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale with a justification. Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with /close with a justification. If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle stale

Send feedback to tektoncd/plumbing.