search

RamenDR / ramen

Apache License 2.0
74 stars 56 forks source link

ci: Retry failed jobs #1626

Open nirs opened 4 weeks ago

nirs commented 4 weeks ago

The e2e and unit tests are not stable enough. Save developers time by retrying failed jobs automatically before failing the build.

Example run - unit tests failed in first attempt, succeeded in the second attempt: https://github.com/RamenDR/ramen/actions/runs/11580608689/job/32239498144

ShyamsundarR commented 3 weeks ago

I'd rather not auto retry so that we can catch and post analysis for failures and enable improving the tests/flakes. This way PRs may get a 1 in 3 (or n) chance of a passing and failures (flake or otherwise) is not analyzed and ignored.

raghavendra-talur commented 6 days ago

I'd rather not auto retry so that we can catch and post analysis for failures and enable improving the tests/flakes. This way PRs may get a 1 in 3 (or n) chance of a passing and failures (flake or otherwise) is not analyzed and ignored.

I agree with Shyam. Let the retries be manual.

nirs commented 6 days ago

I'd rather not auto retry so that we can catch and post analysis for failures and enable improving the tests/flakes. This way PRs may get a 1 in 3 (or n) chance of a passing and failures (flake or otherwise) is not analyzed and ignored.

I agree with Shyam. Let the retries be manual.

Manual retries are fine, but they are not available for people without commit right. We need a way to trigger a rebuild without write permission.