search

defenseunicorns-partnerships / argo-wf-zarf

Zarf package for Argo Workflows
Apache License 2.0
0 stars 0 forks source link

workflow timeout #33

Closed RothAndrew closed 1 month ago

RothAndrew commented 1 month ago

Yesterday I submitted some workflows and ran in to #32. This morning I saw they were still in the cluster still trying to start. We should figure out how to set a timeout so that the argo controller kills them if they aren't finished within a certain time period.

ldgriswold commented 1 month ago

Was the status of these failed? I don't have failed workflows set up to GC, so we could do one of the following:

Change controller.workflowDefaults.spec.ttlStrategy to secondsAfterCompletion instead of secondsAfterSuccess. OR, we could set a controller.workflowDefaults.spec.activeDeadlineSeconds which would limit the amount of time a workflow can run regardless of status (I'd probably set it to something like 3600).

RothAndrew commented 1 month ago

The status was Running, but the actual pods were still 0/3 since Kubernetes wouldn't start them due to the missing secret.

ldgriswold commented 1 month ago

OK then we probably need a activeDeadlineSeconds default param I'll throw one in and PR it