search

tholo / pytest-flake8

pytest plugin to run flake8
Other
117 stars 47 forks source link

Incompatibility with pytest-randomly #26

Open jacquerie opened 7 years ago

jacquerie commented 7 years ago

Minimal reproduction: https://github.com/jacquerie/pytest-flake8-bug

I expect tests to run normally, but this is instead what I see:

% py.test --flake8 tests
================================================================= test session starts =================================================================
platform darwin -- Python 2.7.13, pytest-3.1.3, py-1.4.34, pluggy-0.4.0
Using --randomly-seed=1501011429
rootdir: /Users/jacquerie/Code/pytest-flake8-bug, inifile:
plugins: randomly-1.2.1, flake8-0.8.1
collected 4 items
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/Users/jacquerie/.virtualenvs/flake8/lib/python2.7/site-packages/_pytest/main.py", line 105, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/Users/jacquerie/.virtualenvs/flake8/lib/python2.7/site-packages/_pytest/main.py", line 140, in _main
INTERNALERROR>     config.hook.pytest_collection(session=session)
INTERNALERROR>   File "/Users/jacquerie/.virtualenvs/flake8/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 745, in __call__
INTERNALERROR>     return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR>   File "/Users/jacquerie/.virtualenvs/flake8/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 339, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/Users/jacquerie/.virtualenvs/flake8/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 334, in <lambda>
INTERNALERROR>     _MultiCall(methods, kwargs, hook.spec_opts).execute()
INTERNALERROR>   File "/Users/jacquerie/.virtualenvs/flake8/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 614, in execute
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/Users/jacquerie/.virtualenvs/flake8/lib/python2.7/site-packages/_pytest/main.py", line 150, in pytest_collection
INTERNALERROR>     return session.perform_collect()
INTERNALERROR>   File "/Users/jacquerie/.virtualenvs/flake8/lib/python2.7/site-packages/_pytest/main.py", line 606, in perform_collect
INTERNALERROR>     config=self.config, items=items)
INTERNALERROR>   File "/Users/jacquerie/.virtualenvs/flake8/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 745, in __call__
INTERNALERROR>     return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR>   File "/Users/jacquerie/.virtualenvs/flake8/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 339, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/Users/jacquerie/.virtualenvs/flake8/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 334, in <lambda>
INTERNALERROR>     _MultiCall(methods, kwargs, hook.spec_opts).execute()
INTERNALERROR>   File "/Users/jacquerie/.virtualenvs/flake8/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 614, in execute
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/Users/jacquerie/.virtualenvs/flake8/lib/python2.7/site-packages/pytest_randomly.py", line 119, in pytest_collection_modifyitems
INTERNALERROR>     current_module = item.module
INTERNALERROR> AttributeError: 'Flake8Item' object has no attribute 'module'

============================================================ no tests ran in 0.01 seconds =============================================================
timj commented 7 years ago

I have also come across this. There is also a problem with pytest-random-order. It seems that Flake8Item is breaking the plugin contract and it is assumed that all tests items have some scheme for reporting the module associated with the test. Given that flake8 doesn't have any classes or modules, just files, it might be enough to fake a module attribute that returns the name of the file with __name__. Looking at some of the other bug reports referenced here it seems fixturename is also implied by the pytest plugin contract.

jaraco commented 2 years ago

I wonder if this issue affects other non-module plugins like pytest-checkdocs or pytest-mypy.

jaraco commented 2 years ago

I tried adding pytest-randomly to jaraco.text, which uses a number of other pytest plugins, and it worked just fine:

 jaraco.text main $ tox
python develop-inst-noop: /Users/jaraco/code/main/jaraco.text
python installed: attrs==22.1.0,black==22.6.0,click==8.1.3,coverage==6.4.2,docutils==0.19,filelock==3.7.1,flake8==4.0.1,importlib-metadata==4.12.0,iniconfig==1.1.1,jaraco.context==4.1.2,jaraco.functools==3.5.1,-e git+gh://jaraco/jaraco.text@5572d7a7168dd8643bea1fd50d8d42cdd28b765c#egg=jaraco.text,mccabe==0.6.1,more-itertools==8.13.0,mypy==0.971,mypy-extensions==0.4.3,packaging==21.3,pathspec==0.9.0,pep517==0.12.0,platformdirs==2.5.2,pluggy==1.0.0,py==1.11.0,pycodestyle==2.8.0,pyflakes==2.4.0,pyparsing==3.0.9,pytest==7.1.2,pytest-black==0.3.12,pytest-checkdocs==2.7.1,pytest-cov==3.0.0,pytest-enabler==1.3.0,pytest-flake8==1.1.1,pytest-mypy==0.9.1,pytest-randomly==3.12.0,toml==0.10.2,tomli==2.0.1,typing_extensions==4.3.0,zipp==3.8.1
python run-test-pre: PYTHONHASHSEED='2359061132'
python run-test: commands[0] | pytest
=============================================================================== test session starts ===============================================================================
platform darwin -- Python 3.10.5, pytest-7.1.2, pluggy-1.0.0
cachedir: .tox/python/.pytest_cache
Using --randomly-seed=932733908
rootdir: /Users/jaraco/code/main/jaraco.text, configfile: pytest.ini
plugins: checkdocs-2.7.1, black-0.3.12, randomly-3.12.0, flake8-1.1.1, enabler-1.3.0, mypy-0.9.1, cov-3.0.0
collected 42 items                                                                                                                                                                

jaraco/text/__init__.py ...                                                                                                                                                 [  7%]
jaraco/text/to-dvorak.py s                                                                                                                                                  [  9%]
jaraco/text/__init__.py ..                                                                                                                                                  [ 14%]
jaraco/text/layouts.py s                                                                                                                                                    [ 17%]
jaraco/text/to-dvorak.py .                                                                                                                                                  [ 19%]
jaraco/text/layouts.py .                                                                                                                                                    [ 21%]
jaraco/text/to-qwerty.py s                                                                                                                                                  [ 24%]
jaraco/text/__init__.py .                                                                                                                                                   [ 26%]
jaraco/text/layouts.py .s                                                                                                                                                   [ 31%]
docs/conf.py s                                                                                                                                                              [ 34%]
jaraco/text/__init__.py ..                                                                                                                                                  [ 39%]
. .                                                                                                                                                                         [ 41%]
jaraco/text/__init__.py ....                                                                                                                                                [ 51%]
jaraco/text/to-qwerty.py .                                                                                                                                                  [ 53%]
jaraco/text/__init__.py ..                                                                                                                                                  [ 58%]
docs/conf.py s                                                                                                                                                              [ 60%]
jaraco/text/layouts.py .                                                                                                                                                    [ 63%]
jaraco/text/__init__.py s                                                                                                                                                   [ 65%]
docs/conf.py .                                                                                                                                                              [ 68%]
jaraco/text/__init__.py .                                                                                                                                                   [ 70%]
jaraco/text/to-qwerty.py s                                                                                                                                                  [ 73%]
jaraco/text/to-dvorak.py s                                                                                                                                                  [ 75%]
jaraco/text/__init__.py ......                                                                                                                                              [ 90%]
docs/conf.py .                                                                                                                                                              [ 92%]
jaraco/text/__init__.py s..                                                                                                                                                 [100%]

---------- coverage: platform darwin, python 3.10.5-final-0 ----------
Name                       Stmts   Miss  Cover   Missing
--------------------------------------------------------
docs/conf.py                   7      0   100%
jaraco/text/__init__.py      157      0   100%
jaraco/text/layouts.py         8      0   100%
jaraco/text/to-dvorak.py       3      0   100%
jaraco/text/to-qwerty.py       3      0   100%
--------------------------------------------------------
TOTAL                        178      0   100%

====================================================================================== mypy =======================================================================================

Success: no issues found in 5 source files
========================================================================= 31 passed, 10 skipped in 3.98s ==========================================================================
_____________________________________________________________________________________ summary _____________________________________________________________________________________
  python: commands succeeded
  congratulations :)

I believe this issue can be closed as obsolete.