search

pytest-dev / pytest-timeout

MIT License
216 stars 64 forks source link

Detect when the fail exception was caught in the test #4

Closed pytest-timeout-bot closed 6 years ago

pytest-timeout-bot commented 11 years ago

Original report by Floris Bruynooghe (Bitbucket: flub, GitHub: flub).

Sometimes a test can catch the pytest.fail.Exception itself which makes it look like the test was fine while it did really time out.

One improvement is to use an exception which is a subclass of BaseException instead. Then in the pytest_runtest_call hook catch this exception and change it to a pytest.fail.Exception.

Another improvement is to mark the test item as timed out in the signal handler. Then e.g. pytest_runtest_call call can detect when the test cheated and caught the exception, after which it can fail it anyway with an appropriate message.

pytest-timeout-bot commented 11 years ago

Original comment by Floris Bruynooghe (Bitbucket: flub, GitHub: flub).

Using a different exception object is not that helpful. If the second point is done it covers this much better.

pytest-timeout-bot commented 6 years ago

Original comment by Floris Bruynooghe (Bitbucket: flub, GitHub: flub).

Seems no one cares much