search

Test-More / Test2-Harness

Alternative to Test::Harness
Other
23 stars 26 forks source link

fast-fail mode? #206

Open exodist opened 3 years ago

exodist commented 3 years ago

Might make sense to add a fast-fail mode, where the moment anything happens that makes the test fail, it is killed and reported as failure, not allowed to continue.

Why:

Often at work I find myself running the entire test suite just to get a list of what tests fail. Usually this is because of a new feature or big change. I mainly just want a count of failures, and will then run the tests one by 1 to figure out the issue and fix it. In this case having a failing test continue to stumble does not help, I do not need the extra debugging. In this case a fast-fail would save me a lot of time.

Maybe --fast-fail?

This should probably wait for the 'overlord' branch to be completed and merged, the implementation would be significantly simpler.

toddr commented 3 years ago

doesn't test::more already provide that? https://metacpan.org/pod/Test::More#BAIL_OUT

exodist commented 3 years ago

If you add that to every test yes, but then it is always on. I am talking a global switch for it. I guess you could just pass yath the option to import DieOnFail for every test, BailOnFail is not what we want here cause a bail kills the whole suite, I just want to kill the test that is already known to be failing.

exodist commented 3 years ago

Also DieOnFail is not sufficient, need an ExitOnFail as a lot of failures can happen in eval's.