pytest-dev / pytest-timeout

MIT License
206 stars 63 forks source link

Measure CPU time instead of wallclock time #157

Open link2xt opened 10 months ago

link2xt commented 10 months ago

Currently pytest-timeout uses threading.Timer to measure time. This measures wallclock time, so if the process is paused for 10 minutes and is unpaused later, timeout will immediately expire even if the process did not have a chance to actually do any work. This happens in GitHub Actions CI, which sometimes unschedules processes for a long time as I have observed for Rust testsuite running under nextest: reported time of some tests which normally take less than one second was randomly several minutes. It is ok if CI suspends the workflow, but pytest-timeout should detect this and measure actual time the test was running rather than wallclock time.

flub commented 10 months ago

Whoa, I've never observed GitHub Actions suspending the process before. That's some pretty funky CI behaviour. I've previously declined requests for using the CPU time with "just set your timer higher", but if it really does suspend your process for 10 mins that's a bit of a pain.

How often does this occur? Which runners does it occur on?

link2xt commented 10 months ago

Here is some report of this happening: https://github.com/deltachat/deltachat-core-rust/pull/4066 It likely does not pause the process with Unix signals and such, but does not allocate CPU time to the VM on a hypervisor level.

flub commented 10 months ago

On Wed 01 Nov 2023 at 14:06 -0700, link2xt wrote:

Here is some report of this happening: https://github.com/deltachat/deltachat-core-rust/pull/4066

Did you share the right link? I don't see any CI action logs on that PR

link2xt commented 10 months ago

On Wed 01 Nov 2023 at 14:06 -0700, link2xt wrote: Here is some report of this happening: deltachat/deltachat-core-rust#4066 Did you share the right link? I don't see any CI action logs on that PR

Yes, there is a comment talking about the issue there: https://github.com/deltachat/deltachat-core-rust/pull/4066#issuecomment-1437224156