search

argoproj / argo-workflows

Workflow Engine for Kubernetes
https://argo-workflows.readthedocs.io/
Apache License 2.0
15.08k stars 3.2k forks source link

add "Timeout" option and retry for function `runKubectl` #11965

Open henrywangx opened 1 year ago

henrywangx commented 1 year ago

Summary

add "Timeout" option and retry for function runKubectl.

Use Cases

  1. If the apiserver is unreachable for some reason, runKubectl will hang. We need to set a reasonable timeout and retry after few seconds.

Message from the maintainers:

Love this enhancement proposal? Give it a 👍. We prioritise the proposals with the most 👍.

agilgur5 commented 1 year ago

Potentially a duplicate of #7179. You can also workaround this by using kubectl directly in a script template, for example.

Since #7860 is open, I don't think more will be added to the resource spec, preferring a plugin instead, but patches that do not change the spec could be acceptable.

agilgur5 commented 1 year ago

5549 also mentions using activeDeadlineSeconds as a workaround

henrywangx commented 1 year ago

Potentially a duplicate of #7179. You can also workaround this by using kubectl directly in a script template, for example.

Since #7860 is open, I don't think more will be added to the resource spec, preferring a plugin instead, but patches that do not change the spec could be acceptable.

No, we use the resource, which uses argo controller's runKubectl to create/get k8s object. And we faced the hang problem when it try to get object's status.

agilgur5 commented 1 year ago

I think you misunderstood me. I mentioned a script for workaround purposes.

Otherwise #7179 and #7860 are indeed about resource templates.

terrytangyuan commented 1 year ago

Have you tried the workaround I mentioned in https://github.com/argoproj/argo-workflows/issues/5549#issuecomment-810448427?