Open kaddkaka opened 7 months ago
pytest considers exception/errors in setup/teardown an error (as that shouldnt happen) if it happens in a test, its as expected
i beleive timeout should transition to a new outcome type as a timeout is more similar to a xfail
pytest considers exception/errors in setup/teardown an error (as that shouldnt happen) if it happens in a test, its as expected
i beleive timeout should transition to a new outcome type as a timeout is more similar to a xfail
That sounds reasonable and "Timeout" outcome sounds good. Is there work ongoing or another issue to link this one to?
Not yet
On Fri 22 Mar 2024 at 02:48 -0700, Ronny Pfannschmidt wrote:
i beleive timeout should transition to a new outcome type as a timeout is more similar to a xfail
can plugins define new outcome types?
I believe plugins can set any outcome as long as they handle it appropriately in a pytest_report_teststatus
.
I'm running a smaller testsuite and moste of the tests fail 10 seconds timeout because of slow setup code (currently residing inside the tests). Some of the timeouted tests result in
Error
some inFailure
:Why? Is this intentional?
If I increase my timeout the ERRORs seem to go away and all timeouted tests are listed as FAILUREs.