Using flake8 inflates junit.xml output significantly

[asfaltboy]

tl;dr The junit.xml report generated when running with --flake8 is about x130 (130 times) of the report size without --flake8.

When running py.test on a relatively large project without using --flake8 option, the behavior is "ok". py.test collects about 160 "items" from a project of about 160 modules, and outputs a junit.xml report around 1.4MB in size.

In contrast, when running with --flake8 option, on the same source project, py.test collects about 314 items and outputs a report around 190MB in size.

My environment is:

platform darwin -- Python 2.7.11, pytest-3.0.1, py-1.4.31, pluggy-0.3.1
django settings: my_project.test_settings (from environment variable)
rootdir: /Users/pavel/project_root, inifile: pytest.ini
plugins: cov-2.3.1, django-2.9.1, flake8-0.6, mock-1.2

My pytest.ini is:

[pytest]
DJANGO_SETTINGS_MODULE = my_project.test_settings
flake8-ignore =
    *.py E402 E126 E127 E265 E128 W503 E122 E731 E713 W191 E101 E226
    urls.py ALL
    **/migrations/** ALL
    **/tests/** E501
flake8-max-line-length = 110
addopts =
    --flake8
    --junitxml=reports/junit.xml

Seems that pytest==2.9.2 does not result in the same behaviour. Specifically, if existing compiled pyc/cache files of the modules exist (from the pytest==3.0.1 run), many of the collected items are skipped, resulting in a small xml file. However, if cache/pyc files are cleared, the tests report is again at 190MB. Running without --flake8 again results in a 1.4MB xml.

[tholo]

I would check if maybe you need to add a

[pytest]
norecursedirs = ...

setting to pytest.ini to make sure pytest does not descend into folders it should not?

[asfaltboy]

I'm not sure there's anything I would like to exclude from pytest, just from flake8. And anyhow, why would adding --flake8 extend the collection of tests?

[tholo]

Because it will recurse into e.g. ".tox" folders if you have been running that, and thus add whole instances of standard libraries etc. to the lists in your junit.xml files?

[asfaltboy]

That makes full sense! It's been a while since I had the issue, so no need to keep it open anyway.