Suppressing fixture's contribution to pytest coverage.

[chintal]

I'm trying to see if there is a way to get code executed by a fixture (preferably a specific one via a secondary decorator or keyword argument, or I can live with a way to suppress it globally on fixtures)

I'm writing some tests which operate on constructed objects, where the objects themselves are the subject of the library being tested. The over all flow for the library's functionality looks something like this :

• A collection of parsers operate on an input. In my specific case, it is a essentially a linear state machine, but that isn't important.
• Each of the parsers adds certain information into an 'object' of some sort, which is 'ready' for use after the state machine reaches an end state.
• The object is used by the library to provide bits of information to the user.

When testing this, I can write tests for the first section using restricted strings.

The second section is tested by providing 'real' input examples, and making sure the object is constructed and returned with the same type, and checking that some of the headline numbers within it are correct. This is done by means of a fixture which uses a 'process' command which is part of the same library, and which uses all those elements of the library which made the first section.

The third section is tested by using the same fixture, and making sure that whatever post processing code is there does what it is supposed to be doing.

The problem is that as soon as I include the fixture, all of the code from the first section is used by the fixture itself, and is marked as covered. Using coverage to guide test development for the first section is no longer possible. With literally 5 lines of test code I have 53% coverage.

I would like to make a distinction by stipulating that "test fixtures are helper functions which don't actually test code, and that only things which are specified as tests are tests." As such, any code executed within the fixture shouldn't be counted for coverage, but code within tests that use the fixture should.

I did see the mark.no_cover decorator, but I don't think I can use that to create such a test setup. Is there some available way to implement such coverage counting?

[ionelmc]

Perhaps @nedbat's context feature would support some sort of counting via contexts?

[chintal]

I'm not sure what that feature is, or where to start looking for it...

[jurasofish]

@chintal did you ever find a solution? In exactly the same situation.

[chintal]

@jurasofish No, I did not.

[jurasofish]

rip I'll make an update if I find a way to do it. I suspect it's not too hard

[ionelmc]

Well I guess this would be fixed when I also fix #418

[dweigand]

Hi, we encountered this issue because some of the fixtures in our tests call a large chunk of our code base so that coverage reports are not representative of the true coverage anymore. I've written a workaround and thought you might be interested. It is based upon @ionelmc's comment in https://github.com/pytest-dev/pytest-cov/issues/418#issuecomment-657219659. As a warning, it is not very clean, but it works:

from decorator import decorate
def _disable_pytest_coverage(function, *args, **kw):
    config = args[-1]
    cov = config.pluginmanager.get_plugin("_cov")
    cov.cov_controller.pause()
    result = function(*args, **kw)
    cov.cov_controller.resume()
    return result

def disable_pytest_coverage(f):
    return decorate(f, _disable_pytest_coverage)

decorator is already a dependency of pytest. disable_pytest_coverage inspects arguments of the fixture in order to get access to the Config instance of pytest. Fixtures using this decorator MUST have pytestconfig as last parameter. Usage:

from c3_qccs.qccs_toolbox.helpers import disable_pytest_coverage

@pytest.fixture
@disable_pytest_coverage
def empty_qubit(other, fixtures, ..., pytestconfig):
    return QubitSettings()

[nedbat]

This could be useful for people not using pytest-cov. Are you interested in pursuing an implementation that goes direct to coverage.py?