search

di / pytest-reqs

py.test plugin for checking requirements files
https://pypi.org/p/pytest-reqs
MIT License
25 stars 6 forks source link

Requirements checks are not before other tests #9

Closed jayvdb closed 7 years ago

jayvdb commented 7 years ago

pytest-reqs adds the new checks at the end, so they run last. They should be run before other tests, as the requirement failures will explain any other failures that are related to the versions of dependencies.

jayvdb commented 7 years ago

I have a workaround coming, and probably a fix also.

jayvdb commented 7 years ago

So the workaround is to not add command line option --reqs, but instead create a tests/conftest.py with the following:

from pytest_reqs import pytest_collection_modifyitems as _modifyitems

def pytest_collection_modifyitems(config, session, items):
    config.option.reqs = True
    _modifyitems(config, session, items)
    config.option.reqs = False

This loads them first, as conftest.py is the first 'plugin'. All other plugins are rather random (see https://github.com/pytest-dev/pytest/issues/935).

di commented 7 years ago

Hey @jayvdb, thanks for the bug report. Like you mentioned, it's not currently possible for pytest-reqs to enforce any sort of precedence with regards to other plugins.

Your solution is great though. I just made #10 to make this a little cleaner to do, and add some docs about it for anyone else that would like the same behavior.

jayvdb commented 7 years ago

Note that #10 doesnt really solve this issue, even with the modified conftest.py trick. The requirements checks are still placed last. But since the tests are added in conftest.py, before other plugins, pytest-reorder can see and reorder them.