pytest-dev / pytest-timeout

MIT License
216 stars 64 forks source link

New hook called right before timeout occurs #15

Closed pytest-timeout-bot closed 4 years ago

pytest-timeout-bot commented 8 years ago

Original report by Bruno Oliveira (Bitbucket: nicoddemus, GitHub: nicoddemus).


It would be nice if pytest-timeout supplied a hook that is called right before a timeout occurs:

def pytest_about_to_timeout(config):
    """Called when a timeout is reached and the test suite is about to be aborted.
    """

It could be called as the first thing in dump_stacks.

What do you think @flub? If you like the idea I could work on a PR.

pytest-timeout-bot commented 8 years ago

Original comment by Bruno Oliveira (Bitbucket: nicoddemus, GitHub: nicoddemus).


Just for reference, this is what I've hacked together to use on our internal code base at work:

https://gist.github.com/nicoddemus/9ce9033ab2ba46f11b35a2fc5e5f3ad4

pytest-timeout-bot commented 8 years ago

Original comment by Bruno Oliveira (Bitbucket: nicoddemus, GitHub: nicoddemus).


cc @fabioz

pytest-timeout-bot commented 8 years ago

Original comment by Floris Bruynooghe (Bitbucket: flub, GitHub: flub).


My initial reaction is somewhat cautious, what is the usecase? Problem is that running code after a timeout is just not that reliable. It works well if you're in python-land and can work with the SIGALRM signal but timing out in a general case is just hard. And really given that you can have locks held in the stack while the timeout happen and all sorts of nasty things encouraging people to add even more code at that point seems like a recipe for even more blowing away of your own toes ;-). But I'm not saying no, just asking what the usecase is :-)

To be fair, pretty much everything in pytest-timeout comes with a big caveat of "you better know what you're doing". So maybe adding one extra isn't that big a deal.

pytest-timeout-bot commented 8 years ago

Original comment by Bruno Oliveira (Bitbucket: nicoddemus, GitHub: nicoddemus).


Sorry, I meant to write my use case as well but forgot.

When a timeout occurs in one of our GUI applications, we would like to take a screenshot of the current screen to try to figure out why that test timed out: sometimes this is due to something unexpected happening in a test and a modal dialog being displayed which waits indefinitely for user input. Having a screenshot at that moment would be useful to try to pinpoint which test is causing that because we would be able to see the actual dialog.

I agree with the caveat of "you better know what you're doing" and the hook should mention that. ;)

pytest-timeout-bot commented 5 years ago

Original comment by Floris Bruynooghe (Bitbucket: flub, GitHub: flub).


I'm not against this, but also unlikely to implement this myself... I presume you also no longer need this? Trying out this "on hold" state bitbucket has (not that this will stay much longer on bitbucket, oh well)

pytest-timeout-bot commented 5 years ago

Original comment by Bruno Oliveira (Bitbucket: nicoddemus, GitHub: nicoddemus).


Not at the moment, agree with putting on hold. :+1:

graingert commented 4 years ago

@nicoddemus in your gui application you probably want a 16ms timeout on callbacks from the eventloop, rather than a big global one for the whole test

nicoddemus commented 4 years ago

@graingert what do you mean, posting a 16ms callback to the eventloop and terminate the test if the callback is triggered?

chrahunt commented 4 years ago

Another concrete use case: I have a library that requires multiprocess interaction. The tests spawn subprocesses and bugs often result in the processes hanging, leading to a timeout. It has been really helpful to get the stacks of all the child processes for later inspection, especially for issues that only show up in CI. Example usage here.

flub commented 4 years ago

Closing as no one seems to be working on this. Feel free to open a PR however.