search

avocado-framework / avocado

Avocado is a set of tools and libraries to help with automated testing. One can call it a test framework with benefits. Native tests are written in Python and they follow the unittest pattern, but any executable can serve as a test.
https://avocado-framework.github.io/
Other
345 stars 343 forks source link

Avocado instrumented timeout handling in thread #6060

Open richtja opened 3 weeks ago

richtja commented 3 weeks ago

This commit changes the way how avocado instrumented runner handles test timeouts. For timeout handling, we used to use signals and raising TestInterrupt error. Such solution has an issue that we can't control where in the code the Error will be raised, and it can be handled before it reaches the runner layer. More info about this issue in #6046.

This change removes the signal handling and uses threading instead. Now each test method will be run in a separated thread and this thread will be terminated if timeout is reached. This solution selves the raising error issue and keeps the current test lifecycle untouched.

Reference: #6046

codecov[bot] commented 3 weeks ago

Codecov Report

Attention: Patch coverage is 76.19048% with 5 lines in your changes missing coverage. Please review.

Project coverage is 51.72%. Comparing base (bf9b3d5) to head (1555489). Report is 8 commits behind head on master.

Files with missing lines Patch % Lines
avocado/plugins/runners/avocado_instrumented.py 33.33% 4 Missing :warning:
avocado/core/test.py 93.33% 1 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #6060 +/- ## ========================================== - Coverage 54.45% 51.72% -2.74% ========================================== Files 202 202 Lines 21885 21888 +3 ========================================== - Hits 11917 11321 -596 - Misses 9968 10567 +599 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

clebergnu commented 2 weeks ago

It would be interesting to run this with external test suites, such as the tests on QEMU or sosreport. I can attempt that as part of my review.