Coverage missing for QThreads

[hhslepicka]

New Issue to separate the discussion on #582.

Here is a gist file for the test:
https://gist.github.com/hhslepicka/3153c9c2ac27dee4398754dce6a11fea

Environment:

• macOS High Sierra - 10.13.5
• Tested with Python 2.7 and 3.6 (Anaconda)
• Coverage: 4.5.1 with C extension

Dependencies:

• Python 2.7 or 3.6 (I tested with both)
• PyQt5 & Qt5... any version...

Some useful information:

(py36) slepicka@public-81-101:~/sandbox/covtest  $ coverage --version
Coverage.py, version 4.5.1 with C extension
Documentation at https://coverage.readthedocs.io

(py36) slepicka@dhcp-swlan-public-81-101:~/sandbox/covtest  $ coverage run test.py
Counter:  56

(py36) slepicka@dhcp-swlan-public-81-101:~/sandbox/covtest  $ coverage report -m
Name      Stmts   Miss  Cover   Missing
---------------------------------------
test.py      24      4    83%   13-15, 18

Documentation for QThread in case it is useful: http://doc.qt.io/qt-5/qthread.html Just let me know if you need additional info from my system.

[hhslepicka]

From @nedbat's post in another thread.

Asked on the PyQt list: Any way to run code before each QThread?.

From there: an example of monkey-patching QThread to add debugging support: http://die-offenbachs.homelinux.org:48888/hg/eric/file/87b1626eb49b/DebugClients/Python/ThreadExtension.py#l361

[enkore]

Even when explicitly registering sys.settrace, it doesn't work. Am I missing something?

https://gist.github.com/enkore/14c53e9af79e9bbaf66478115605633f

[earonesty]

I tried sys.settrace(threading._trace_hook) and it seems to work fine... coverage is reported on all threads.

Important to note for @enkore : your test code will only track coverage of the counter. Coverage starts at the next function call. So your sleep will not be covered (for example). Easiest way to fix this is to have run call self._run.... where the real code is.

Maybe all coveragepy needs is to enshrine this by adding a : coverage.thread_start() function. Then users of exotic threading system can just call that function.

[Dennis-van-Gils]

I solved this by writing a custom decorator to decorate your own methods with that are afflicted by this problem. Disclaimer: I am a Python enthusiast and not an expert, but it seems to work all right for me.

Paste this code to the top of your module:

# Code coverage tools 'coverage' and 'pytest-cov' don't seem to correctly trace 
# code which is inside methods called from within QThreads, see
# https://github.com/nedbat/coveragepy/issues/686
# To mitigate this problem, I use a custom decorator '@coverage_resolve_trace' 
# to be hung onto those method definitions. This will prepend the decorated
# method code with 'sys.settrace(threading._trace_hook)' when a code
# coverage test is detected. When no coverage test is detected, it will just
# pass the original method untouched.
import sys
import threading
from functools import wraps

running_coverage = 'coverage' in sys.modules
if running_coverage: print("\nCode coverage test detected\n")

def coverage_resolve_trace(fn):
    @wraps(fn)
    def wrapped(*args, **kwargs):
        if running_coverage: sys.settrace(threading._trace_hook)
        fn(*args, **kwargs)
    return wrapped    

And remember to decorate your problematic method with it, e.g.

@coverage_resolve_trace
def my_method():
    ....

I hope my solution is of use to others, too.

[Czaki]

I have developed such pytest fixture:

@pytest.fixture()
def cover_qthreads(monkeypatch, qtbot):
    from qtpy.QtCore import QThread

    base_constructor = QThread.__init__

    def run_with_trace(self):  # pragma: no cover
        """
        QThread.run but adding execution to sys.settrace when measuring coverage.

        See https://github.com/nedbat/coveragepy/issues/686#issuecomment-634932753
        and `init_with_trace`. When running QThreads during testing, we monkeypatch
        the QThread constructor and run methods with traceable equivalents.
        """
        if 'coverage' in sys.modules:
            # https://github.com/nedbat/coveragepy/issues/686#issuecomment-634932753
            sys.settrace(threading._trace_hook)
        self._base_run()

    def init_with_trace(self, *args, **kwargs):
        """Constructor for QThread adding tracing for coverage measurements.

        Functions running in QThreads don't get measured by coverage.py, see
        https://github.com/nedbat/coveragepy/issues/686. Therefore, we will
        monkeypatch the constructor to add to the thread to `sys.settrace` when
        we call `run` and `coverage` is in `sys.modules`.
        """
        base_constructor(self, *args, **kwargs)
        self._base_run = self.run
        self.run = partial(run_with_trace, self)

    monkeypatch.setattr(QThread, '__init__', init_with_trace)

It monkeypatch run function to init coverage.

[ThomasST0]

Based on previous fixture, I did it for QThreadPool

@fixture(autouse=True)
def cover_qthreadpool(monkeypatch, qtbot):
  from PySide6.QtCore import QThreadPool

  base_constructor = QThreadPool.globalInstance().start

  def run_with_trace(self):  # pragma: no cover
      """
      QThread.run but adding execution to sys.settrace when measuring coverage.

      See https://github.com/nedbat/coveragepy/issues/686#issuecomment-634932753
      and `init_with_trace`. When running QThreads during testing, we monkeypatch
      the QThread constructor and run methods with traceable equivalents.
      """
      if "coverage" in sys.modules:
          # https://github.com/nedbat/coveragepy/issues/686#issuecomment-634932753
          sys.settrace(threading._trace_hook)
      self._base_run()

  def _start(worker, *args, **kwargs):
      """Constructor for QThread adding tracing for coverage measurements.

      Functions running in QThreads don't get measured by coverage.py, see
      https://github.com/nedbat/coveragepy/issues/686. Therefore, we will
      monkeypatch the constructor to add to the thread to `sys.settrace` when
      we call `run` and `coverage` is in `sys.modules`.
      """
      worker._base_run = worker.run
      worker.run = partial(run_with_trace, worker)
      return base_constructor(worker, *args, **kwargs)

  monkeypatch.setattr(QThreadPool.globalInstance(), "start", _start)