`pylint` + `pylint-pytest`: pylint crashed with a ``AstroidError`` and with the following stacktrace

stdedos commented 1 year ago

Bug description

Using to "facilitate" pylint + pytest, I receive:

Traceback (most recent call last):
  File "/p/.venv/lib/python3.9/site-packages/pylint/lint/pylinter.py", line 798, in _lint_file
    check_astroid_module(module)
  File "/p/.venv/lib/python3.9/site-packages/pylint/lint/pylinter.py", line 1067, in check_astroid_module
    retval = self._check_astroid_module(
  File "/p/.venv/lib/python3.9/site-packages/pylint/lint/pylinter.py", line 1117, in _check_astroid_module
    walker.walk(node)
  File "/p/.venv/lib/python3.9/site-packages/pylint/utils/ast_walker.py", line 91, in walk
    callback(astroid)
  File "/p/.venv/lib/python3.9/site-packages/pylint_pytest/checkers/fixture.py", line 125, in visit_module
    ret = pytest.main(
  File "/p/.venv/lib/python3.9/site-packages/_pytest/config/__init__.py", line 148, in main
    config = _prepareconfig(args, plugins)
  File "/p/.venv/lib/python3.9/site-packages/_pytest/config/__init__.py", line 329, in _prepareconfig
    config = pluginmanager.hook.pytest_cmdline_parse(
  File "/p/.venv/lib/python3.9/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/p/.venv/lib/python3.9/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/p/.venv/lib/python3.9/site-packages/pluggy/_callers.py", line 55, in _multicall
    gen.send(outcome)
  File "/p/.venv/lib/python3.9/site-packages/_pytest/helpconfig.py", line 103, in pytest_cmdline_parse
    config: Config = outcome.get_result()
  File "/p/.venv/lib/python3.9/site-packages/pluggy/_result.py", line 60, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/p/.venv/lib/python3.9/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/p/.venv/lib/python3.9/site-packages/_pytest/config/__init__.py", line 1058, in pytest_cmdline_parse
    self.parse(args)
  File "/p/.venv/lib/python3.9/site-packages/_pytest/config/__init__.py", line 1346, in parse
    self._preparse(args, addopts=addopts)
  File "/p/.venv/lib/python3.9/site-packages/_pytest/config/__init__.py", line 1248, in _preparse
    self.hook.pytest_load_initial_conftests(
  File "/p/.venv/lib/python3.9/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/p/.venv/lib/python3.9/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/p/.venv/lib/python3.9/site-packages/pluggy/_callers.py", line 60, in _multicall
    return outcome.get_result()
  File "/p/.venv/lib/python3.9/site-packages/pluggy/_result.py", line 60, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/p/.venv/lib/python3.9/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/p/.venv/lib/python3.9/site-packages/_pytest/config/__init__.py", line 1125, in pytest_load_initial_conftests
    self.pluginmanager._set_initial_conftests(
  File "/p/.venv/lib/python3.9/site-packages/_pytest/config/__init__.py", line 557, in _set_initial_conftests
    self._try_load_conftest(anchor, namespace.importmode, rootpath)
  File "/p/.venv/lib/python3.9/site-packages/_pytest/config/__init__.py", line 574, in _try_load_conftest
    self._getconftestmodules(anchor, importmode, rootpath)
  File "/p/.venv/lib/python3.9/site-packages/_pytest/config/__init__.py", line 603, in _getconftestmodules
    mod = self._importconftest(conftestpath, importmode, rootpath)
  File "/p/.venv/lib/python3.9/site-packages/_pytest/config/__init__.py", line 651, in _importconftest
    self.consider_conftest(mod)
  File "/p/.venv/lib/python3.9/site-packages/_pytest/config/__init__.py", line 732, in consider_conftest
    self.register(conftestmodule, name=conftestmodule.__file__)
  File "/p/.venv/lib/python3.9/site-packages/_pytest/config/__init__.py", line 489, in register
    ret: Optional[str] = super().register(plugin, name)
  File "/p/.venv/lib/python3.9/site-packages/pluggy/_manager.py", line 114, in register
    self._verify_hook(hook, hookimpl)
  File "/p/.venv/lib/python3.9/site-packages/pluggy/_manager.py", line 232, in _verify_hook
    raise PluginValidationError(
pluggy._manager.PluginValidationError: Plugin '/p/path/to/tests/conftest.py' for hook 'pytest_collectstart'
hookimpl definition: _pytest_collectstart(_collector)
Argument(s) {'_collector'} are declared in the hookimpl but can not be found in the hookspec

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/p/.venv/lib/python3.9/site-packages/pylint/lint/pylinter.py", line 763, in _lint_files
    self._lint_file(fileitem, module, check_astroid_module)
  File "/p/.venv/lib/python3.9/site-packages/pylint/lint/pylinter.py", line 800, in _lint_file
    raise astroid.AstroidError from e
astroid.exceptions.AstroidError

Configuration

No response

Command used

bash -c 'test -d .venv && . .venv/bin/activate ; pylint-ignore "$@"' -

--output-format=pylint_gitlab.GitlabCodeClimateReporter:GitlabCodeClimateReporter-pylint.json,colorized
--score=no
--disable=R,C,useless-suppression

... and then files of the repo.

(i.e., called via pre-commit)

(PS: add the following information on the ~/.cache/pylint/pylint-crash-2023-02-15-14-17-14.txt file :pray:)

Pylint output

tappylint-ignore-ltp...............................................................................Failed
- hook id: pylint-ignore-ltp
- exit code: 1

Exception on node <Module.tests.helpers l.0 at 0x7fa956b4e400> in file 'p/path/to/tests/helpers/__init__.py'
...traceback...
************* Module tests.helpers.__init__
path/to/tests/helpers/__init__.py:1:0: F0002: path/to/tests/helpers/__init__.py: Fatal error while checking 'path/to/tests/helpers/__init__.py'. Please open an issue in our bug tracker so we address this. There is a pre-filled template that you can use in '~/.cache/pylint/pylint-crash-2023-02-15-14-16-58.txt'. (astroid-error)
Exception on node <Module.tests.test_sanity l.0 at 0x7f715021d2b0> in file 'p/path/to/tests/test_sanity.py'
...traceback...
************* Module tests.test_sanity
path/to/tests/test_sanity.py:1:0: F0002: path/to/tests/test_sanity.py: Fatal error while checking 'path/to/tests/test_sanity.py'. Please open an issue in our bug tracker so we address this. There is a pre-filled template that you can use in '~/.cache/pylint/pylint-crash-2023-02-15-14-16-59.txt'. (astroid-error)
Exception on node <Module.tests.helpers.plotly l.0 at 0x7f5f4d866df0> in file 'p/path/to/tests/helpers/plotly.py'
...traceback...
************* Module tests.helpers.plotly
path/to/tests/helpers/plotly.py:1:0: F0002: path/to/tests/helpers/plotly.py: Fatal error while checking 'path/to/tests/helpers/plotly.py'. Please open an issue in our bug tracker so we address this. There is a pre-filled template that you can use in '~/.cache/pylint/pylint-crash-2023-02-15-14-17-04.txt'. (astroid-error)
Exception on node <Module.tests.test_balance l.0 at 0x7f1743e6dfa0> in file 'p/path/to/tests/test_balance.py'
...traceback...
************* Module tests.test_balance
path/to/tests/test_balance.py:1:0: F0002: path/to/tests/test_balance.py: Fatal error while checking 'path/to/tests/test_balance.py'. Please open an issue in our bug tracker so we address this. There is a pre-filled template that you can use in '~/.cache/pylint/pylint-crash-2023-02-15-14-17-01.txt'. (astroid-error)
Exception on node <Module.tests.test_parsers l.0 at 0x7f43423cf3d0> in file 'p/path/to/tests/test_parsers.py'
...traceback...
************* Module tests.test_parsers
path/to/tests/test_parsers.py:1:0: F0002: path/to/tests/test_parsers.py: Fatal error while checking 'path/to/tests/test_parsers.py'. Please open an issue in our bug tracker so we address this. There is a pre-filled template that you can use in '~/.cache/pylint/pylint-crash-2023-02-15-14-17-08.txt'. (astroid-error)
Exception on node <Module.tests.conftest l.0 at 0x7feecf1c4fa0> in file 'p/path/to/tests/conftest.py'
...traceback...
************* Module tests.conftest
path/to/tests/conftest.py:1:0: F0002: path/to/tests/conftest.py: Fatal error while checking 'path/to/tests/conftest.py'. Please open an issue in our bug tracker so we address this. There is a pre-filled template that you can use in '~/.cache/pylint/pylint-crash-2023-02-15-14-16-58.txt'. (astroid-error)
************* Module tests.helpers.paths
path/to/tests/helpers/paths.py:1:0: F0002: path/to/tests/helpers/paths.py: Fatal error while checking 'path/to/tests/helpers/paths.py'. Please open an issue in our bug tracker so we address this. There is a pre-filled template that you can use in '~/.cache/pylint/pylint-crash-2023-02-15-14-16-58.txt'. (astroid-error)
Exception on node <Module.tests l.0 at 0x7f5c1055d100> in file 'p/path/to/tests/__init__.py'
...traceback...
************* Module tests.__init__
path/to/tests/__init__.py:1:0: F0002: path/to/tests/__init__.py: Fatal error while checking 'path/to/tests/__init__.py'. Please open an issue in our bug tracker so we address this. There is a pre-filled template that you can use in '~/.cache/pylint/pylint-crash-2023-02-15-14-16-59.txt'. (astroid-error)

pylint-ignore-idp...............................................................................Failed
- hook id: pylint-ignore-idp
- exit code: 1

Exception on node <Module.tests.conftest l.0 at 0x7ff85182efa0> in file 'p/htap/to/tests/conftest.py'
...traceback...
************* Module tests.conftest
htap/to/tests/conftest.py:1:0: F0002: htap/to/tests/conftest.py: Fatal error while checking 'htap/to/tests/conftest.py'. Please open an issue in our bug tracker so we address this. There is a pre-filled template that you can use in '~/.cache/pylint/pylint-crash-2023-02-15-14-17-14.txt'. (astroid-error)
************* Module tests.helpers.paths
htap/to/tests/helpers/paths.py:1:0: F0002: htap/to/tests/helpers/paths.py: Fatal error while checking 'htap/to/tests/helpers/paths.py'. Please open an issue in our bug tracker so we address this. There is a pre-filled template that you can use in '~/.cache/pylint/pylint-crash-2023-02-15-14-17-14.txt'. (astroid-error)
Exception on node <Module.tests.test_parsers l.0 at 0x7f955cfe0fa0> in file 'p/htap/to/tests/test_parsers.py'
...traceback...
************* Module tests.test_parsers
htap/to/tests/test_parsers.py:1:0: F0002: htap/to/tests/test_parsers.py: Fatal error while checking 'htap/to/tests/test_parsers.py'. Please open an issue in our bug tracker so we address this. There is a pre-filled template that you can use in '~/.cache/pylint/pylint-crash-2023-02-15-14-17-16.txt'. (astroid-error)
Exception on node <Module.tests l.0 at 0x7f0d3337fca0> in file 'p/htap/to/tests/__init__.py'
...traceback...
************* Module tests.__init__
htap/to/tests/__init__.py:1:0: F0002: htap/to/tests/__init__.py: Fatal error while checking 'htap/to/tests/__init__.py'. Please open an issue in our bug tracker so we address this. There is a pre-filled template that you can use in '~/.cache/pylint/pylint-crash-2023-02-15-14-17-14.txt'. (astroid-error)
Exception on node <Module.tests.test_sanity l.0 at 0x7f359fe04fa0> in file 'p/htap/to/tests/test_sanity.py'
...traceback...
************* Module tests.test_sanity
htap/to/tests/test_sanity.py:1:0: F0002: htap/to/tests/test_sanity.py: Fatal error while checking 'htap/to/tests/test_sanity.py'. Please open an issue in our bug tracker so we address this. There is a pre-filled template that you can use in '~/.cache/pylint/pylint-crash-2023-02-15-14-17-21.txt'. (astroid-error)
Exception on node <Module.tests.helpers l.0 at 0x7f5c4130a100> in file 'p/htap/to/tests/helpers/__init__.py'
...traceback...
************* Module tests.helpers.__init__
htap/to/tests/helpers/__init__.py:1:0: F0002: htap/to/tests/helpers/__init__.py: Fatal error while checking 'htap/to/tests/helpers/__init__.py'. Please open an issue in our bug tracker so we address this. There is a pre-filled template that you can use in '~/.cache/pylint/pylint-crash-2023-02-15-14-17-14.txt'. (astroid-error)

Expected behavior

Not-a-stack-trace

Pylint version

pylint 2.16.0
astroid 2.14.1
Python 3.9.16 (main, Dec  7 2022, 01:11:51) 
[GCC 9.4.0]

OS / Environment

Ubuntu 20.04

Additional dependencies

No response

Pierre-Sassoulas commented 1 year ago

It seems it's a problem inside pylint-pytest, but right now it's unclear who's maintaining it so let's keep this issue opened until it's clearer.

stdedos commented 1 year ago

Should ~~pytest~~ pylint capture these at the top level?

I like the "maybe-pylint-bug" template - but I think on that case ~~pytest~~ pylint may have been able to say "plugin xyz exploded. pls report to plugin-maintainer. I'll continue without it." or something along those lines?

Pierre-Sassoulas commented 1 year ago

There's three libs at work here:

pylint (here)
pytest (not here but I can also answer on this one if you open an issue in their repo)
pylint-pytest (not here, maybe in another org soon but right now it's unclear who's maintaining it)

It seems this bug is happening outside of pylint, if you want something to change in pytest you're going to need to open an issue in pytest, if you want something to change in pylint-pytest there's no clear maintainer ship right now, but we plan on maintaining it one way or the other in the midterm see https://github.com/PyCQA/meta/issues/56.

stdedos commented 1 year ago

Sorry, sorry. I meant pylint :facepalm:

Pierre-Sassoulas commented 1 year ago

Ha, right. No it's voluntary to not catch the error, we need the stacktrace and a clear fail when something goes wrong (so we get an issue opened and the users have information available).

stdedos commented 1 year ago

I was thinking that, "if you did all that", but then just unloaded the plugin and continue, it would give pylints issue tracker less downstream issues. But sure, I agree that explicit > implicit, so let's let it be.

I'm planning to touch pylint-pytest soon with https://github.com/reverbc/pylint-pytest/pull/22 - so might as well give this a better look, on top of context.

pylint-dev / pylint