search

akuity / kargo

Application lifecycle orchestration
https://kargo.akuity.io/
Apache License 2.0
1.39k stars 114 forks source link

Multiple Verifications running for a Stage #2142

Open jessesuen opened 3 weeks ago

jessesuen commented 3 weeks ago

Description

Stages seem to be allowing multiple Verifications to be run at the same time. Each coincided with a promotion.

image 
$ k get ar
NAME                                     STATUS    AGE
dev.01j02dz4tbp75sf7zmvgkw3d3j.9e5f6ab   Failed    16h
dev.01j02hycta7xsjz7xczamptk6c.9d167fb   Failed    14h
dev.01j03w1h1njwvj432nbh8m7xvk.a72c52c   Running   158m
dev.01j03wxxntp3a3xm5pbpdzzzkd.bd13218   Running   142m
dev.01j03z1dzstgnx5tw85ty3apxr.97465d1   Running   105m

I feel that we should only be allowing one at a time to happen.

We should also decide how Promotion should affect currently running Verifications. e.g. should it:

Steps to Reproduce

Promote something else while a verification is already running.

Version

v0.7.0
krancour commented 3 weeks ago

Possibly a duplicate of #2128?

hiddeco commented 3 weeks ago

block the promotion until verification is completed?

I feel this would (again) steer more towards verification being an element of promotion, which based on the further discussion in #2128 does not appear to be true. In addition, it hypothetically opens a gap to e.g. quickly issue re-verifications to block a promotion from ever starting.

Given this, I am inclined to say we should wait for a running verification to complete (and maybe terminate it if there is new current Freight), before triggering a new verification.

krancour commented 3 weeks ago

which based on the further discussion in https://github.com/akuity/kargo/issues/2128...

fwiw, my thoughts there were in no way definitive/final.

Let's hash this out f2f when time permits.