Closed goyalankit closed 2 years ago
Hi @goyalankit . Thanks for the issue.
I think it makes sense, WDYT @kubeflow/wg-training-leads
I think this is reasonable improvement. @goyalankit Feel free to cut a PR and assign to us to help review
Thanks for this
Let's keep it open to track cherry-pick
/reopen
@Jeffwan: Reopened this issue.
In the current implementation, the
runPolicy.BackoffLimit
is only applicable onOnFailure
andAlways
restart policies. I was wondering if there's a reason it's not supported on theExitCode
restart policy. Often in the case of OOM, the job exits with an exit code of 137 which is rightfully a retriable error. However, it will keep restarting indefinitely since the policy is not covered by theBackOffLimit
. The behavior could be to honor the backOffLimit if present, else it keeps retrying indefinitely?I am happy to submit a PR if you think this is a reasonable change.