search

reframe-hpc / reframe

A powerful Python framework for writing and running portable regression tests and benchmarks for HPC systems.
https://reframe-hpc.readthedocs.org
BSD 3-Clause "New" or "Revised" License
213 stars 100 forks source link

Do not retry any test if total failures have exceeded a certain limit #3139

Open vkarak opened 4 months ago

vkarak commented 4 months ago

This is essentially a variation of the --max-retries option where the tests will be retried only if the total number of failures is below a certain threshold. This is different from the --maxfail option which will stop the session once the total number of failures is reached. This request would run all the tests once and will eventually decide whether it is worth retrying them or not.