Azure / pytest-azurepipelines

Plugin for pytest that makes it simple to work with Azure Pipelines
MIT License
113 stars 35 forks source link

pytest_configure() updates pytest & coverage config #83

Open MoreGreenForests opened 1 year ago

MoreGreenForests commented 1 year ago

Took awhile testing this:

I am running with python 3.10, pytest 7.2.1 and pytest-cov 4.0.0 & pytest-aurepipeline 1.0.4.

Problem: I tried pytest-azurepipelines with default configs and it worked fine. Working with local testing runs and cloud I wanted to build a clean setup to run test and coverage reports to a tests/reports sub-directory instead of outputing to the main directory when running with a local IDE.

When I attempted to update the configuration it took awhile to figure out the problem. I was using a pyproject.toml file with addopts overrides in an ini section. Without azurepipelines installed it worked locally as expected. When installed I see my xml path is overwritten with the coverage default and full file path instead of the relative path I was putting in. Printing out the config I was able to see the elements that this function is the culprit.

@pytest.hookimpl(tryfirst=True)
def pytest_configure(config):
    xmlpath = config.getoption("--nunitxml")
    if not xmlpath:
        config.option.nunit_xmlpath = DEFAULT_PATH

    # ensure coverage creates xml format
    if config.pluginmanager.has_plugin("pytest_cov"):
        if config.option.cov_report.get("xml") is None:
            config.option.cov_report["xml"] = os.path.normpath(
                os.path.abspath(os.path.expanduser(os.path.expandvars(DEFAULT_COVERAGE_PATH)))
            )
        if "html" not in config.option.cov_report:
            config.option.cov_report["html"] = None

Could remove this tryfirst=True and we should be fine. When I print out the config in the pytest header summary I can see it already gets overwritten. Given this library should likely not be setting the config but receiving and managing the inline css and so forth this is not desirable as it is checking for the config.option.cov_report.get("xml") first before it is set so it is likely always None and overrides a custom config to the default config path. If it should be run first in this library than it should just be called first in order. This runs it first overall within pytest testing run.

I have a workaround in copying the function into a conftest.py file and resetting it back without the tryfirst=True before the pytest_sessionfinish() function is called.