search

pylint-dev / pylint-django

Pylint plugin for improving code analysis for when using Django
Other
591 stars 117 forks source link

pylint-django calls ast_from_module_name without catching AstroidImportError #399

Open jonholdsworth opened 1 year ago

jonholdsworth commented 1 year ago

Bug description

Possibly related to: https://github.com/pylint-dev/pylint/issues/8554

Pylint consistently crashes across all versions including latest, on one particular file, throwing 'AstroidError' mentioning 'inference_tip.py' and '_inference_tip_cached'.

Configuration

No response

Command used

pylint <directory>

Pylint output

I'm finding out if I'm allowed to post the actual code.

The error is:

pylint crashed with a ``AstroidError`` and with the following stacktrace:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/astroid/inference_tip.py", line 33, in _inference_tip_cached
    result = _cache[func, node]
KeyError: (<function infer_key_classes at 0x7ff9268d2c20>, <Call l.20 at 0x7ff921fd10f0>)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/pylint/checkers/utils.py", line 1371, in safe_infer
    value = next(infer_gen)
  File "/usr/local/lib/python3.10/site-packages/astroid/nodes/node_ng.py", line 171, in infer
    yield from self._infer(context=context, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/astroid/decorators.py", line 143, in raise_if_nothing_inferred
    yield next(generator)
  File "/usr/local/lib/python3.10/site-packages/astroid/decorators.py", line 112, in wrapped
    for res in _func(node, context, **kwargs):
  File "/usr/local/lib/python3.10/site-packages/astroid/bases.py", line 177, in _infer_stmts
    for inf in stmt.infer(context=context):
  File "/usr/local/lib/python3.10/site-packages/astroid/nodes/node_ng.py", line 161, in infer
    results = list(self._explicit_inference(self, context, **kwargs))
  File "/usr/local/lib/python3.10/site-packages/astroid/inference_tip.py", line 40, in _inference_tip_cached
    result = _cache[func, node] = list(func(*args, **kwargs))
  File "/usr/local/lib/python3.10/site-packages/pylint_django/transforms/foreignkey.py", line 120, in infer_key_classes
    MANAGER.ast_from_module_name(module_name)
  File "/usr/local/lib/python3.10/site-packages/astroid/manager.py", line 239, in ast_from_module_name
    raise e
  File "/usr/local/lib/python3.10/site-packages/astroid/manager.py", line 184, in ast_from_module_name
    found_spec = self.file_from_module_name(modname, context_file)
  File "/usr/local/lib/python3.10/site-packages/astroid/manager.py", line 292, in file_from_module_name
    raise value.with_traceback(None)  # pylint: disable=no-member
astroid.exceptions.AstroidImportError: Failed to import module projects.models with error:
No module named projects.models.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/pylint/lint/pylinter.py", line 811, in _lint_file
    check_astroid_module(module)
  File "/usr/local/lib/python3.10/site-packages/pylint/lint/pylinter.py", line 1085, in check_astroid_module
    retval = self._check_astroid_module(
  File "/usr/local/lib/python3.10/site-packages/pylint/lint/pylinter.py", line 1135, in _check_astroid_module
    walker.walk(node)
  File "/usr/local/lib/python3.10/site-packages/pylint/utils/ast_walker.py", line 94, in walk
    self.walk(child)
  File "/usr/local/lib/python3.10/site-packages/pylint/utils/ast_walker.py", line 91, in walk
    callback(astroid)
  File "/usr/local/lib/python3.10/site-packages/pylint_plugin_utils/__init__.py", line 58, in __call__
    self.augmentation_func(Chain(self.old_method, node), node)
  File "/usr/local/lib/python3.10/site-packages/pylint_plugin_utils/__init__.py", line 115, in __call__
    chain()
  File "/usr/local/lib/python3.10/site-packages/pylint_plugin_utils/__init__.py", line 67, in __call__
    self.old_method(self.node)
  File "/usr/local/lib/python3.10/site-packages/pylint_plugin_utils/__init__.py", line 58, in __call__
    self.augmentation_func(Chain(self.old_method, node), node)
  File "/usr/local/lib/python3.10/site-packages/pylint_plugin_utils/__init__.py", line 115, in __call__
    chain()
  File "/usr/local/lib/python3.10/site-packages/pylint_plugin_utils/__init__.py", line 67, in __call__
    self.old_method(self.node)
  File "/usr/local/lib/python3.10/site-packages/pylint_plugin_utils/__init__.py", line 58, in __call__
    self.augmentation_func(Chain(self.old_method, node), node)
  File "/usr/local/lib/python3.10/site-packages/pylint_plugin_utils/__init__.py", line 115, in __call__
    chain()
  File "/usr/local/lib/python3.10/site-packages/pylint_plugin_utils/__init__.py", line 67, in __call__
    self.old_method(self.node)
  File "/usr/local/lib/python3.10/site-packages/pylint_plugin_utils/__init__.py", line 58, in __call__
    self.augmentation_func(Chain(self.old_method, node), node)
  File "/usr/local/lib/python3.10/site-packages/pylint_plugin_utils/__init__.py", line 115, in __call__
    chain()
  File "/usr/local/lib/python3.10/site-packages/pylint_plugin_utils/__init__.py", line 67, in __call__
    self.old_method(self.node)
  File "/usr/local/lib/python3.10/site-packages/pylint/checkers/classes/class_checker.py", line 876, in visit_classdef
    self._check_bases_classes(node)
  File "/usr/local/lib/python3.10/site-packages/pylint/checkers/classes/class_checker.py", line 2096, in _check_bases_classes
    unimplemented_abstract_methods(node, is_abstract).items(),
  File "/usr/local/lib/python3.10/site-packages/pylint/checkers/utils.py", line 972, in unimplemented_abstract_methods
    inferred = safe_infer(obj)
  File "/usr/local/lib/python3.10/site-packages/pylint/checkers/utils.py", line 1375, in safe_infer
    raise AstroidError from e
astroid.exceptions.AstroidError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/pylint/lint/pylinter.py", line 775, in _lint_files
    self._lint_file(fileitem, module, check_astroid_module)
  File "/usr/local/lib/python3.10/site-packages/pylint/lint/pylinter.py", line 813, in _lint_file
    raise astroid.AstroidError from e
astroid.exceptions.AstroidError

Expected behavior

Pylint would just warn about code findings and not crash.

Pylint version

pylint 2.17.4 (got same result in 2.17.5 which I now don't seem to be able to install) astroid 2.15.5 Python 3.10.11 (main, May 23 2023, 13:58:30) [GCC 10.2.1 20210110]

OS / Environment

Docker Debian GNU/Linux 11

Additional dependencies

No response

jonholdsworth commented 1 year ago

We managed to fix this via config: with the following .pylintrc the error went away. (There is no propriety information in this config so we are happy to share it.) It looks like the problem was specific to our setup. Please feel free to close this ticket if you wish, I will not complain.

[MASTER]
load-plugins=pylint_django
ignore=features, migrations
django-settings-module=config.settings

[REPORTS]
output-format=colorized

[FORMAT]
max-line-length=120

[MESSAGES CONTROL]
disable=
    RP0101,
    RP0204,
    RP0401,
    RP0402,
    RP0701,
    RP0801,
    W0707,
    C0209,
    R0022,
    R1735,
    W0640,
    R1721,
    W0236,
    W0719,
    C0415,
    R1729,
    R1732,
    W0236,
    R1724,
    R1720,
    W1514,
    R1720,
    attribute-defined-outside-init,
    fixme,
    invalid-name,
    locally-disabled,
    missing-docstring,
    similarities,
    too-few-public-methods,
    too-many-ancestors,
    too-many-lines,
    no-member,
    arguments-differ,
    unused-argument,
    too-many-arguments,
    too-many-instance-attributes,
    line-too-long,
    unnecessary-pass,
    cyclic-import,

[TYPECHECK]
generated-members=_meta,object,objects.request,kwargs,[a-z]+_set,DoesNotExist

[DESIGN]
max-parents=13
DanielNoord commented 1 year ago

@jacobtylerwalls Is this related to our latest changes to that cache?

jacobtylerwalls commented 1 year ago

I don't know yet. I'm planning to take a look this week.

jacobtylerwalls commented 1 year ago

Not related to the cache. (I should do a little refactor to avoid the KeyError from a cache miss showing up in the tracebacks like this.) EDIT: done, see pylint-dev/astroid#2205

It's just that safe_infer() doesn't catch AstroidImportError. (Or, that pylint-django doesn't catch it in their transform.)

jacobtylerwalls commented 1 year ago

pylint-django should catch and deal with AstroidImportError when calling ast_from_module_name. Should we also guard against it safe_infer()? I think it's better to guard against errors you reasonably expect, so probably not.

carlio commented 1 year ago

@jacobtylerwalls Hi there, I'm the one behind pylint-django, I'll look into this.

Do you know what specifically changed in your .pylintrc which made the problem go away?

jacobtylerwalls commented 1 year ago

Sorry, I'm not the reporter! But from this:

astroid.exceptions.AstroidImportError: Failed to import module projects.models with error:
No module named projects.models.

it seems like the change in the rc file made projects.models discoverable somehow.

carlio commented 1 year ago

Yes sorry, I picked the wrong autocomplete after typing @j :-)

@jonholdsworth Do you know what specifically changed in your .pylintrc which made the problem go away?

jonholdsworth commented 1 year ago

My apologies, I should have posted the diff in the first place. It's from bitbucket so I'll have to paste this manually.

Hope this helps!

Changes were:


[MASTER]
load-plugins=pylint_django
ignore=features, migrations
+ django-settings-module=config.settings
[REPORTS]
output-format=colorized
[FORMAT]
max-line-length=120
[MESSAGES CONTROL]
disable=
    RP0101,
+   RP0204,
    RP0401,
    RP0402,
    RP0701,
    RP0801,
+   W0707,
+   C0209,
+   R0022,
+   R1735,
+   W0640,
+   R1721,
+   W0236,
+   W0719,
+   C0415,
+   R1729,
+   R1732,
+   W0236,
+   R1724,
+   R1720,
+   W1514,
+   R1720,
    attribute-defined-outside-init,
    fixme,
    invalid-name,
    locally-disabled,
-    locally-enabled,
    missing-docstring,
-     no-self-use,
-     redefined-variable-type,
    similarities,
    too-few-public-methods,
    too-many-ancestors,
    too-many-lines,
    no-member,
    arguments-differ,
-     bad-continuation,
    unused-argument,
    too-many-arguments,
    too-many-instance-attributes,
    line-too-long,
    unnecessary-pass,
    cyclic-import,
[TYPECHECK]
generated-members=_meta,object,objects.request,kwargs,[a-z]+_set,DoesNotExist
[DESIGN]
max-parents=13