CI to stop if http connection failures occur

[hvgazula]

Should we add conditions in the workflow to quit cleanly (but with a failed flag) if some important tests fail, thus saving time? Or maybe (occurred at the time of writing this), should we add the -x flag to pytest?

pytest -x           # stop after first failure
pytest --maxfail=2  # stop after two failures

[hvgazula]

another thought: how about setting the order of tests? https://pytest-dev.github.io/pytest-order/stable/index.html although it doesn't work for python > 3.10

[hvgazula]

Should we add conditions in the workflow to quit cleanly (but with a failed flag) if some important tests fail, thus saving time? Or maybe (occurred at the time of writing this), should we add the -x flag to pytest?

pytest -x           # stop after first failure
pytest --maxfail=2  # stop after two failures

Running some connection checks on github itself will save time and money on aws

[hvgazula]

@satra Would you prefer a workflow to run all tests (reports failed tests in one go at the end) or stop as soon as it encounters a first failure?

[satra]

it depends on the test-suite. iterative stoppage is very demanding. so easier to see which tests have failed in one go and fix all of them.