Option to retry N times until test passes

[vt-alt]

Perhaps I first misfiled feature request on pytest itself https://github.com/pytest-dev/pytest/issues/11508

What's the problem this feature will solve?

Repeating test after success increases probability of whole test run failure. This is what pytest-repeat does now.

There are situations when cost of false positive failure is high and cost of fixing a particular failing test (to be race free) is even higher (due to lack of human resources).

Good solutions it seems is just to repeat a test after failure, decreasing probability of false positive failure.

Describe the solution you'd like

ctest have option like this

  --repeat until-pass:<n>      = Allow each test to run up to <n> times in order to pass

This could be good for pytest to support something similar to help with 'probabilistic' tests.

For example workflow could have pytest run with something like --repeat-until-pass 3 greatly lowering probability of false positive test failures.

[vt-alt]

Closing because of pytest-rerunfailures.