Python testfiles are being loaded.

[SanketDG]

Thanks for submitting an issue!

Here's a quick checklist in what to include:

• [x] Include a detailed description of the bug or suggestion

I have a codebase where I need to test the contents of python files. Unfortunately, this test files are syntactically invalid, and they are being collected which causes them to fail. I do not want to collect them since they are test files.

• [x] pip list of the virtual environment you are using

alabaster (0.7.7)
ansicolor (0.2.4)
apipkg (1.4)
appdirs (1.4.0)
astroid (1.4.8)
autoflake (0.6.6)
autopep8 (1.2.4)
Babel (2.2.0)
bleach (1.4.2)
chardet (2.3.0)
cib (0.2.2, /home/sanket/cib)
click (6.6)
cmakelint (1.3.4.1)
coala (0.9.0.dev99999999999999, /home/sanket/coala)
coala-decorators (0.2.1)
coala-utils (0.4.9)
colorama (0.3.7)
coverage (4.0.3)
cppclean (0.9)
cpplint (1.3.0)
decorator (4.0.9)
docopt (0.6.2)
docutils (0.12)
eradicate (0.1.6)
execnet (1.4.1)
flake8 (3.0.4)
flake8-polyfill (1.0.1)
Flask (0.11.1)
future (0.15.2)
guess-language-spirit (0.5.2)
html-linter (0.3.0)
html5lib (0.9999999)
isort (4.2.5)
itsdangerous (0.24)
Jinja2 (2.8)
language-check (0.8)
lazy-object-proxy (1.2.2)
libclang-py3 (0.2)
mando (0.3.3)
MarkupSafe (0.23)
mccabe (0.5.2)
munkres3 (1.0.5.5)
nltk (3.2.1)
pager (3.3)
pep8 (1.7.0)
pip (7.0.3)
prompt-toolkit (1.0.7)
proselint (0.3.6)
py (1.4.31)
pycodestyle (2.0.0)
pydocstyle (1.0.0)
pyflakes (1.2.3)
Pygments (2.1.3)
pylint (1.6.4)
PyPrint (0.2.4)
pytest (2.8.7)
pytest-cov (2.2.1)
pytest-env (0.6.0)
pytest-mock (1.1)
pytest-timeout (1.0.0)
pytest-xdist (1.14)
pytz (2016.3)
PyYAML (3.11)
radon (1.4.2)
readme-renderer (0.7.0)
requests (2.11.0)
requests-mock (0.7.0)
restructuredtext-lint (0.14.3)
restview (2.6.1)
selenium (3.0.0b2)
setuptools (25.2.0)
six (1.10.0)
snowballstemmer (1.2.1)
Sphinx (1.3.5)
sphinx-rtd-theme (0.1.9)
template-remover (0.1.9)
termcolor (1.1.0)
validators (0.10)
vim-vint (0.3.9)
vulture (0.10)
wcwidth (0.1.7)
Werkzeug (0.11.10)
wheel (0.24.0)
wrapt (1.10.8)
yamllint (1.3.2)
yapf (0.11.1)

• [x] pytest and operating system versions

$ py.test -v

This is pytest version 2.8.7, imported from /home/sanket/.virtualenvs/coala/lib/python3.4/site-packages/pytest.py
setuptools registered plugins:
  pytest-env-0.6.0 at /home/sanket/.virtualenvs/coala/lib/python3.4/site-packages/pytest_env/plugin.py
  pytest-cov-2.2.1 at /home/sanket/.virtualenvs/coala/lib/python3.4/site-packages/pytest_cov/plugin.py
  pytest-timeout-1.0.0 at /home/sanket/.virtualenvs/coala/lib/python3.4/site-packages/pytest_timeout.py
  pytest-mock-1.1 at /home/sanket/.virtualenvs/coala/lib/python3.4/site-packages/pytest_mock.py
  pytest-xdist-1.14 at /home/sanket/.virtualenvs/coala/lib/python3.4/site-packages/xdist/looponfail.py
  pytest-xdist-1.14 at /home/sanket/.virtualenvs/coala/lib/python3.4/site-packages/xdist/boxed.py
  pytest-xdist-1.14 at /home/sanket/.virtualenvs/coala/lib/python3.4/site-packages/xdist/plugin.py

[nicoddemus]

Could you describe your problem in more details please? It's not clear to me what you mean by "I have a codebase where I need to test the contents of python files. Unfortunately, this test files are syntactically invalid, and they are being collected which causes them to fail."

[SanketDG]

Um, so I have python files like these:

def empty():
    """
       this is heavy

    :param x:
    :return:"""return 88

As you can see, this is invalid python and would return a compilation error if compiled.

I have "test files" like this i.e. files I just read... I dont want them to be collected. Is that clear?

[nicoddemus]

Thanks!

How are you naming these test files? You can control which files pytest tries to collect as test files using the python_files configuration variable. Docs: http://doc.pytest.org/en/latest/customize.html?highlight=python_files#confval-python_files

[SanketDG]

Um they are in bad_file*.py format.

You can control which files pytest tries to collect as test files using the python_files configuration variable.

This is what I am more confused about. In my setup.cfg:

[pytest]
norecursedirs =
    .git
    dist
    build
    venv
    .env
python_files = *Test.py
python_classes = *Test
python_functions = *_test
timeout = 35
addopts = --color=yes --ignore=test-requirements.txt
env =
    PYTHONHASHSEED=0
# PYTHONHASHSEED=0 is required to use same hashes in pytests-xdist's workers

[coverage:run]
branch = True
cover_pylib = False
source =
    .
omit =
    tests/*
    .ci/*
    setup.py

[coverage:report]
show_missing = True

So python_files shouldn't pick that up.

Here are the two errors:

_______________________ ERROR collecting tests/documentation/test_files/DocumentationStyleBear/bad_file2.py ________________________
../.virtualenvs/coala/lib/python3.4/site-packages/py/_path/local.py:650: in pyimport
    __import__(modname)
E     File "/home/sanket/coala-bears/tests/documentation/test_files/DocumentationStyleBear/bad_file2.py", line 1
E       def x:
E            ^
E   SyntaxError: invalid syntax
_______________________ ERROR collecting tests/documentation/test_files/DocumentationStyleBear/bad_file3.py ________________________
../.virtualenvs/coala/lib/python3.4/site-packages/py/_path/local.py:650: in pyimport
    __import__(modname)
E     File "/home/sanket/coala-bears/tests/documentation/test_files/DocumentationStyleBear/bad_file3.py", line 6
E       """
E          this is heavy
E   
E       :param x:
E       :return:"""return 88
E          
E                       
E   
E                
E                       ^
E   SyntaxError: invalid syntax

[nicoddemus]

Hmm that's strange, can you post a reproducible example?

[SanketDG]

Sure

$ git clone git@github.com:coala-analyzer/coala-bears.git
$ git checkout sanketdg/docformatbear
$ pip install -r test-requirements.txt
$ py.test

[The-Compiler]

That branch doesn't seem to exist in your repo.

[SanketDG]

@The-Compiler sorry, I have updated the same.

[The-Compiler]

I can't reproduce this. tests/documentation/DocumentationStyleBearTest.py is the only thing collected in tests/documentation here.

Could you maybe share a full terminal session where you reproduce this, ideally from zero and with a clean virtualenv?

[SanketDG]

Interesting, looks like a git clean -dfx worked, and it doesn't seem to collect it anymore.

[nicoddemus]

Hmm still it's a little strange.

Anyway, can we close this now?

[SanketDG]

Sure, but after pushing the CI still fails. https://circleci.com/gh/coala-analyzer/coala-bears/2863

[The-Compiler]

I can reproduce this by doing pytest --doctest-modules tests. Can you actually reproduce it without --doctest-modules like you said above?

[SanketDG]

@The-Compiler, yep this is it! It doesn't work with --doctest-modules!

[The-Compiler]

Okay - I'll let someone else take a look then, as I have no idea about pytest's doctest integration.

By the way: To make things easier, in the future please try actually reproducing your issue with the given reproduction steps - an example which doesn't actually reproduce the problem is quite useless and only makes people spend time on trying to see why they can't reproduce it instead of fixing the bug :wink: