search

pylint-dev / pylint-pytest

A Pylint plugin to suppress pytest-related false positives.
https://pypi.org/project/pylint-pytest/
MIT License
14 stars 3 forks source link

RecursionError when setting jobs != 1 #56

Closed danielsamuels closed 3 months ago

danielsamuels commented 3 months ago

Describe the bug When configuring the number of Pytest workers to anything other than 1, a recursion error is seen and the linting task fails to complete.

To Reproduce

astroid==3.1.0
pylint-django==2.5.5
pylint-plugin-utils==0.8.2
pylint-pytest==1.1.7
pylint==3.1.0

Folder structure src\namespace\module\submodule\settings__init__.py

File content

from namespace.module.submodule.event import log_event
from namespace.module.submodule.util import get_ip_address

def log_auth_event(msg, request):
    log_event("auth", dict(msg, remote_addr=get_ip_address(request)))

def log_user_logged_in(sender, user, request, **kwargs):  # pylint: disable=unused-argument
    log_auth_event(dict(username=user.username, action="login", result="success"), request)

def log_user_logged_out(sender, user, request, **kwargs):  # pylint: disable=unused-argument
    if not user:
        return
    log_auth_event(dict(username=user.username, action="logout", result="success"), request)

def log_user_login_failed(sender, credentials, request, **kwargs):  # pylint: disable=unused-argument
    log_auth_event(
        dict(username=credentials["username"], action="login", result="failure"),
        request,
    )

pylint output with the plugin

Exception on node <FunctionDef.log_user_logged_in l.9 at 0x257d06d9150> in file 'C:\dev\project_name\src\br\bls\audit\auth.py'
Traceback (most recent call last):
  File "C:\dev\project_name\venv\Lib\site-packages\pylint\utils\ast_walker.py", line 96, in walk
    callback(astroid)
  File "C:\dev\project_name\venv\Lib\site-packages\pylint\checkers\variables.py", line 1557, in leave_functiondef
    self._check_is_unused(
  File "C:\dev\project_name\venv\Lib\site-packages\pylint\checkers\variables.py", line 2730, in _check_is_unused
    self._check_unused_arguments(name, node, stmt, argnames, nonlocal_names)
  File "C:\dev\project_name\venv\Lib\site-packages\pylint\checkers\variables.py", line 2853, in _check_unused_arguments
    self.add_message("unused-argument", args=name, node=stmt, confidence=confidence)
  File "C:\dev\project_name\venv\Lib\site-packages\pylint_pytest\checkers\fixture.py", line 314, in patch_add_message
    FixtureChecker._original_add_message(self, msgid, line, node, args, confidence, col_offset)
  File "C:\dev\project_name\venv\Lib\site-packages\pylint_pytest\checkers\fixture.py", line 314, in patch_add_message
    FixtureChecker._original_add_message(self, msgid, line, node, args, confidence, col_offset)
  File "C:\dev\project_name\venv\Lib\site-packages\pylint_pytest\checkers\fixture.py", line 314, in patch_add_message
    FixtureChecker._original_add_message(self, msgid, line, node, args, confidence, col_offset)
  [Previous line repeated 976 more times]
  File "C:\dev\project_name\venv\Lib\site-packages\pylint_pytest\checkers\fixture.py", line 290, in patch_add_message
    and _can_use_fixture(node.parent.parent)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\dev\project_name\venv\Lib\site-packages\pylint_pytest\utils.py", line 113, in _can_use_fixture
    if function.name.startswith("test_") or function.name.endswith("_test"):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RecursionError: maximum recursion depth exceeded while calling a Python object
concurrent.futures.process._RemoteTraceback:
"""
Traceback (most recent call last):
  File "C:\dev\project_name\venv\Lib\site-packages\pylint\lint\pylinter.py", line 830, in _check_file
    check_astroid_module(ast_node)
  File "C:\dev\project_name\venv\Lib\site-packages\pylint\lint\pylinter.py", line 1016, in check_astroid_module
    retval = self._check_astroid_module(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\dev\project_name\venv\Lib\site-packages\pylint\lint\pylinter.py", line 1068, in _check_astroid_module
    walker.walk(node)
  File "C:\dev\project_name\venv\Lib\site-packages\pylint\utils\ast_walker.py", line 94, in walk
    self.walk(child)
  File "C:\dev\project_name\venv\Lib\site-packages\pylint\utils\ast_walker.py", line 96, in walk
    callback(astroid)
  File "C:\dev\project_name\venv\Lib\site-packages\pylint\checkers\variables.py", line 1557, in leave_functiondef
    self._check_is_unused(
  File "C:\dev\project_name\venv\Lib\site-packages\pylint\checkers\variables.py", line 2730, in _check_is_unused
    self._check_unused_arguments(name, node, stmt, argnames, nonlocal_names)
  File "C:\dev\project_name\venv\Lib\site-packages\pylint\checkers\variables.py", line 2853, in _check_unused_arguments
    self.add_message("unused-argument", args=name, node=stmt, confidence=confidence)
  File "C:\dev\project_name\venv\Lib\site-packages\pylint_pytest\checkers\fixture.py", line 314, in patch_add_message
    FixtureChecker._original_add_message(self, msgid, line, node, args, confidence, col_offset)
  File "C:\dev\project_name\venv\Lib\site-packages\pylint_pytest\checkers\fixture.py", line 314, in patch_add_message
    FixtureChecker._original_add_message(self, msgid, line, node, args, confidence, col_offset)
  File "C:\dev\project_name\venv\Lib\site-packages\pylint_pytest\checkers\fixture.py", line 314, in patch_add_message
    FixtureChecker._original_add_message(self, msgid, line, node, args, confidence, col_offset)
  [Previous line repeated 976 more times]
  File "C:\dev\project_name\venv\Lib\site-packages\pylint_pytest\checkers\fixture.py", line 290, in patch_add_message
    and _can_use_fixture(node.parent.parent)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\dev\project_name\venv\Lib\site-packages\pylint_pytest\utils.py", line 113, in _can_use_fixture
    if function.name.startswith("test_") or function.name.endswith("_test"):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RecursionError: maximum recursion depth exceeded while calling a Python object

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Program Files\Python311\Lib\concurrent\futures\process.py", line 256, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\concurrent\futures\process.py", line 205, in _process_chunk
    return [fn(*args) for args in chunk]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\concurrent\futures\process.py", line 205, in <listcomp>
    return [fn(*args) for args in chunk]
            ^^^^^^^^^
  File "C:\dev\project_name\venv\Lib\site-packages\pylint\lint\parallel.py", line 79, in _worker_check_single_file
    _worker_linter.check_single_file_item(file_item)
  File "C:\dev\project_name\venv\Lib\site-packages\pylint\lint\pylinter.py", line 739, in check_single_file_item
    self._check_file(self.get_ast, check_astroid_module, file)
  File "C:\dev\project_name\venv\Lib\site-packages\pylint\lint\pylinter.py", line 832, in _check_file
    raise astroid.AstroidError from e
astroid.exceptions.AstroidError
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\dev\project_name\venv\Scripts\pylint.exe\__main__.py", line 7, in <module>
  File "C:\dev\project_name\venv\Lib\site-packages\pylint\__init__.py", line 34, in run_pylint
    PylintRun(argv or sys.argv[1:])
  File "C:\dev\project_name\venv\Lib\site-packages\pylint\lint\run.py", line 211, in __init__
    linter.check(args)
  File "C:\dev\project_name\venv\Lib\site-packages\pylint\lint\pylinter.py", line 678, in check
    check_parallel(
  File "C:\dev\project_name\venv\Lib\site-packages\pylint\lint\parallel.py", line 153, in check_parallel
    for (
  File "C:\Program Files\Python311\Lib\concurrent\futures\process.py", line 602, in _chain_from_iterable_of_lists
    for element in iterable:
  File "C:\Program Files\Python311\Lib\concurrent\futures\_base.py", line 619, in result_iterator
    yield _result_or_cancel(fs.pop())
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\concurrent\futures\_base.py", line 317, in _result_or_cancel
    return fut.result(timeout)
           ^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\concurrent\futures\_base.py", line 456, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\concurrent\futures\_base.py", line 401, in __get_result
    raise self._exception
astroid.exceptions.AstroidError

Expected behavior The plugin should result in the same output as when running in single-threaded mode.

Additional context I'm on a 16-core machine, and I've tried with jobs=0, 1, and 2. jobs=1 is fine; everything else breaks.

stdedos commented 3 months ago

Hello @danielsamuels!

This looks like https://github.com/pylint-dev/pylint-pytest/pull/29. Would you try https://github.com/pylint-dev/pylint-pytest/releases/tag/v2.0.0a0?

danielsamuels commented 3 months ago

Yup, that works. What's the current release timeline for that?

stdedos commented 3 months ago

Sadly, it's a bit stale 😕

My OSS time is pretty limited, and, with the new pytest I'm hoping to debug one very obscure issue I'd like fixed (which I haven't found a way to make it "straightforward" to rapidly test it)

However, I would suggest to "depend" on it >=2.0.0a0. I am not planning any more extensive changes for this release, and I will follow semantic versioning.

I will close this issue now (to mark it appropriately), but of course feel free to continue the discussion 🙃