search

pylint-dev / pylint

It's not just a linter that annoys you!
https://pylint.readthedocs.io/en/latest/
GNU General Public License v2.0
5.33k stars 1.14k forks source link

Pylint crashes with astroid.exceptions.AstroidImportError #2995

Closed danihodovic closed 5 years ago

danihodovic commented 5 years ago

Steps to reproduce

Install wagtail==2.5.1 and lint the following code:

from wagtail.core.models import Page

class BlogIndexPage(Page):
    content_panels = Page.content_panels  # noqa

Current behavior

pylint crashes

$ pylint blog/models2.py
Traceback (most recent call last):
  File "/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/astroid/__init__.py", line 91, in _inference_tip_cached
    return iter(_cache[func, node])
KeyError: (<function infer_key_classes at 0x7f37c6971378>, <Call l.245 at 0x7f37c5dbd550>)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/dani/repos/jobs_aggregator/venv/bin/pylint", line 11, in <module>
    sys.exit(run_pylint())
  File "/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/pylint/__init__.py", line 20, in run_pylint
    Run(sys.argv[1:])
  File "/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/pylint/lint.py", line 1628, in __init__
    linter.check(args)
  File "/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/pylint/lint.py", line 943, in check
    self._do_check(files_or_modules)
  File "/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/pylint/lint.py", line 1075, in _do_check
    self.check_astroid_module(ast_node, walker, rawcheckers, tokencheckers)
  File "/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/pylint/lint.py", line 1158, in check_astroid_module
    walker.walk(ast_node)
  File "/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/pylint/utils.py", line 1303, in walk
    self.walk(child)
  File "/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/pylint/utils.py", line 1300, in walk
    cb(astroid)
  File "/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/pylint_plugin_utils/__init__.py", line 57, in augment_func
    augmentation(chain, node)
  File "/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/pylint_plugin_utils/__init__.py", line 148, in do_suppress
    chain()
  File "/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/pylint_plugin_utils/__init__.py", line 56, in chain
    old_method(node)
  File "/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/pylint_plugin_utils/__init__.py", line 57, in augment_func
    augmentation(chain, node)
  File "/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/pylint_plugin_utils/__init__.py", line 148, in do_suppress
    chain()
  File "/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/pylint_plugin_utils/__init__.py", line 56, in chain
    old_method(node)
  File "/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/pylint_plugin_utils/__init__.py", line 57, in augment_func
    augmentation(chain, node)
  File "/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/pylint_plugin_utils/__init__.py", line 148, in do_suppress
    chain()
  File "/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/pylint_plugin_utils/__init__.py", line 56, in chain
    old_method(node)
  File "/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/pylint_plugin_utils/__init__.py", line 57, in augment_func
    augmentation(chain, node)
  File "/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/pylint_plugin_utils/__init__.py", line 148, in do_suppress
    chain()
  File "/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/pylint_plugin_utils/__init__.py", line 56, in chain
    old_method(node)
  File "/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/pylint/checkers/classes.py", line 729, in visit_classdef
    self._check_bases_classes(node)
  File "/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/pylint/checkers/classes.py", line 1444, in _check_bases_classes
    unimplemented_abstract_methods(node, is_abstract).items(),
  File "/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/pylint/checkers/utils.py", line 782, in unimplemented_abstract_methods
    infered = safe_infer(obj)
  File "/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/pylint/checkers/utils.py", line 1066, in safe_infer
    value = next(inferit)
  File "/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/astroid/decorators.py", line 131, in raise_if_nothing_inferred
    yield next(generator)
  File "/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/astroid/decorators.py", line 95, in wrapped
    res = next(generator)
  File "/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/astroid/bases.py", line 137, in _infer_stmts
    for inferred in stmt.infer(context=context):
  File "/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/astroid/node_classes.py", line 348, in infer
    return self._explicit_inference(self, context, **kwargs)
  File "/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/astroid/__init__.py", line 93, in _inference_tip_cached
    result = func(*args, **kwargs)
  File "/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/pylint_django/transforms/foreignkey.py", line 96, in infer_key_classes
    MANAGER.ast_from_module_name(module_name)
  File "/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/astroid/manager.py", line 178, in ast_from_module_name
    raise e
  File "/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/astroid/manager.py", line 129, in ast_from_module_name
    found_spec = self.file_from_module_name(modname, context_file)
  File "/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/astroid/manager.py", line 222, in file_from_module_name
    raise value
astroid.exceptions.AstroidImportError: Failed to import module contenttypes.models with error:
No module named contenttypes.models.

Expected behavior

pylint --version output

$ pylint --version
pylint 2.3.1
astroid 2.2.5
Python 3.7.3 (default, Apr  9 2019, 04:56:51)
[GCC 8.3.0]
PCManticore commented 5 years ago

Hey @danihodovic Can you try with the latest unreleased code from both pylint and astroid? Testing with those versions I'm getting the following:

a.py:1:0: C0111: Missing module docstring (missing-docstring)
a.py:3:0: C0111: Missing class docstring (missing-docstring)
a.py:3:0: R0901: Too many ancestors (8/7) (too-many-ancestors)
danihodovic commented 5 years ago 
pip install git+https://github.com/PyCQA/pylint.git
pip install git+https://github.com/PyCQA/astroid.git
$ pylint blog/models.py
Traceback (most recent call last):
  File "/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/pylint_plugin_utils/__init__.py", line 3, in <module>
    from pylint.utils import UnknownMessage
ImportError: cannot import name 'UnknownMessage' from 'pylint.utils' (/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/pylint/utils/__init__.py)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/dani/repos/jobs_aggregator/venv/bin/pylint", line 11, in <module>
    load_entry_point('pylint==2.4.0.dev0', 'console_scripts', 'pylint')()
  File "/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/pylint/__init__.py", line 20, in run_pylint
    Run(sys.argv[1:])
  File "/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/pylint/lint.py", line 1655, in __init__
    linter.load_plugin_modules(plugins)
  File "/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/pylint/lint.py", line 662, in load_plugin_modules
    module = modutils.load_module_from_name(modname)
  File "/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/astroid/modutils.py", line 202, in load_module_from_name
    return load_module_from_modpath(dotted_name.split("."), path, use_sys)
  File "/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/astroid/modutils.py", line 245, in load_module_from_modpath
    module = imp.load_module(curname, mp_file, mp_filename, mp_desc)
  File "/usr/lib/python3.7/imp.py", line 244, in load_module
    return load_package(name, filename)
  File "/usr/lib/python3.7/imp.py", line 216, in load_package
    return _load(spec)
  File "<frozen importlib._bootstrap>", line 696, in _load
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/pylint_django/__init__.py", line 6, in <module>
    from pylint_django import plugin
  File "/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/pylint_django/plugin.py", line 3, in <module>
    from pylint_plugin_utils import get_checker
  File "/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/pylint_plugin_utils/__init__.py", line 5, in <module>
    from pylint.utils import UnknownMessageError as UnknownMessage
ImportError: cannot import name 'UnknownMessageError' from 'pylint.utils' (/home/dani/repos/jobs_aggregator/venv/lib/python3.7/site-packages/pylint/utils/__init__.py)
richkirsch commented 5 years ago

I'm also experiencing this error. I can confirm OP's steps.

gyermolenko commented 5 years ago

I couldn't reproduce initiall issue. Here are my outputs :

# w/o wagtail==2.5.1 installed
importerr.py:1:0: C0111: Missing module docstring (missing-docstring)
importerr.py:1:0: E0401: Unable to import 'wagtail.core.models' (import-error)
importerr.py:3:0: C0111: Missing class docstring (missing-docstring)
importerr.py:3:0: R0903: Too few public methods (0/2) (too-few-public-methods)

# with wagtail
$ pylint blog/models.py
************* Module models
blog/models.py:1:0: C0111: Missing module docstring (missing-docstring)
blog/models.py:3:0: C0111: Missing class docstring (missing-docstring)
blog/models.py:3:0: R0901: Too many ancestors (8/7) (too-many-ancestors)

-----------------------------------
Your code has been rated at 0.00/10

$ pylint --version
pylint 2.4.0-dev0
astroid 2.3.0
Python 3.7.3 (default, May 20 2019, 17:30:06)
[Clang 10.0.1 (clang-1001.0.46.4)]
gyermolenko commented 5 years ago

@danihodovic could you please post your full pip list output? (which pylint plugins do you have?)

richkirsch commented 5 years ago

I'm using pylint==2.3.1 and astroid==2.2.5 for the record. Good to know it seems to be fixed in future releases.

danihodovic commented 5 years ago 
appdirs (1.4.3)
argh (0.26.2)
astroid (2.2.5)
atomicwrites (1.3.0)
attrs (19.1.0)
awscli (1.16.181)
backcall (0.1.0)
beautifulsoup4 (4.7.1)
black (19.3b0)
bleach (2.1.4)
blis (0.2.4)
boto3 (1.9.171)
botocore (1.12.171)
cachetools (3.1.1)
certifi (2019.6.16)
chardet (3.0.4)
Click (7.0)
cloudflare (2.3.0)
colorama (0.3.9)
coverage (4.5.3)
croniter (0.3.30)
CurrencyConverter (0.13.9)
cymem (2.0.2)
dateparser (0.7.1)
decorator (4.4.0)
Django (2.2.3)
django-anymail (6.1.0)
django-crispy-forms (1.7.2)
django-csp (3.5)
django-debug-toolbar (1.11)
django-extensions (2.1.9)
django-health-check (3.10.2)
django-htmlmin (0.11.0)
django-meta (1.5.1)
django-mjml (0.7.0)
django-model-utils (3.2.0)
django-modelcluster (4.4)
django-money (0.15)
django-query-parameters (0.2.3)
django-redis (4.10.0)
django-rq (2.1.0)
django-storages (1.7.1)
django-stubs (0.12.1)
django-taggit (0.24.0)
django-tellme (0.6.5)
django-treebeard (4.3)
django-user-agents (0.3.2)
djangorestframework (3.9.4)
docopt (0.6.2)
docutils (0.14)
draftjs-exporter (2.1.6)
en-core-web-sm (2.1.0)
factory-boy (2.12.0)
Faker (1.0.7)
furl (2.0.0)
future (0.17.1)
google-api-python-client (1.7.9)
google-auth (1.6.3)
google-auth-httplib2 (0.0.3)
gunicorn (19.9.0)
html2text (2018.1.9)
html5lib (1.0.1)
htmlmin (0.1.12)
httplib2 (0.13.0)
idna (2.8)
importlib-metadata (0.18)
ipython (7.5.0)
ipython-genutils (0.2.0)
isort (4.3.20)
jedi (0.13.3)
jmespath (0.9.4)
jsonlines (1.2.0)
jsonschema (3.0.1)
lazy-object-proxy (1.4.1)
Markdown (2.6.11)
mccabe (0.6.1)
more-itertools (7.0.0)
murmurhash (1.0.2)
mypy (0.711)
mypy-extensions (0.4.1)
nh-currency (1.0.1)
numpy (1.16.4)
oauth2client (4.1.3)
orderedmultidict (1.0)
packaging (19.0)
parso (0.4.0)
pathtools (0.1.2)
pexpect (4.7.0)
pickleshare (0.7.5)
Pillow (5.4.1)
pip (9.0.1)
pkg-resources (0.0.0)
plac (0.9.6)
pluggy (0.12.0)
praw (6.3.1)
prawcore (1.0.1)
preshed (2.0.1)
prompt-toolkit (2.0.9)
psycopg2 (2.8.2)
ptipython (1.0.1)
ptpython (2.0.4)
ptyprocess (0.6.0)
pudb (2019.1)
py (1.8.0)
py-moneyed (0.8.0)
pyasn1 (0.4.5)
pyasn1-modules (0.2.5)
pygal (2.4.0)
Pygments (2.4.2)
pylint (2.3.1)
pylint-django (2.0.9)
pylint-plugin-utils (0.5)
pyparsing (2.4.0)
pyrsistent (0.15.2)
PySocks (1.7.0)
pytest (5.0.0)
pytest-cov (2.7.1)
pytest-django (3.5.1)
python-dateutil (2.8.0)
python-http-client (3.1.0)
pytz (2019.1)
PyYAML (5.1)
redis (3.2.1)
regex (2019.6.8)
requests (2.22.0)
responses (0.10.6)
rq (1.0)
rq-scheduler (0.9)
rsa (4.0)
s3transfer (0.2.1)
sendgrid (6.0.5)
setuptools (41.0.1)
six (1.12.0)
soupsieve (1.9.1)
spacy (2.1.4)
sqlparse (0.3.0)
srsly (0.0.7)
stripe (2.30.0)
text-unidecode (1.2)
thinc (7.0.4)
toml (0.10.0)
tqdm (4.32.2)
traitlets (4.3.2)
typed-ast (1.4.0)
typing-extensions (3.7.2)
tzlocal (1.5.1)
ua-parser (0.8.0)
Unidecode (1.1.0)
update-checker (0.16)
uritemplate (3.0.0)
urllib3 (1.25.3)
urwid (2.0.1)
user-agents (2.0)
wagtail (2.5.1)
wagtail-markdown (0.5)
wagtail-metadata (2.0.1)
Wand (0.5.6)
wasabi (0.2.2)
watchdog (0.9.0)
wcwidth (0.1.7)
webencodings (0.5.1)
websocket-client (0.56.0)
Werkzeug (0.15.4)
Willow (1.1)
wrapt (1.11.2)
zipp (0.5.1)
gyermolenko commented 5 years ago

well, I couldn't reproduce with 

$ pylint --version
pylint 2.3.1
astroid 2.2.5
Python 3.7.4 (default, Aug  6 2019, 10:43:15)
[Clang 11.0.0 (clang-1100.0.20.17)]

and this requirements.txt

pylint==2.3.1
wagtail==2.5.1
wagtail-markdown==0.5
wagtail-metadata==2.0.1
pylint-django==2.0.9
pylint-plugin-utils==0.5

Without reproduction scenario I don't think it is likely to be fixed.

danihodovic commented 5 years ago

I think this could be related to #2937

danihodovic commented 5 years ago

I've replicated this error in a Docker image https://github.com/danihodovic/replicate-pylint-error

git clone git@github.com:danihodovic/replicate-pylint-error.git
cd replicate-pylint-error
docker build -t pylint-error .
docker run pylint-error

also found some other related issues:

danihodovic commented 5 years ago

I've fixed my problem with https://github.com/wagtail/wagtail/pull/5538

It turns out that wagtail is not compatible with the pylint-django plugin, but I'm hoping that the above fix gets merged. For more details see that PR.

PCManticore commented 5 years ago

Thanks @danihodovic for the update! Given this was caused by pylint-django, it's probably safe to close this one for now.