Pylint crash when it imports module dkim "KeyError: <class '_ast.Sub'>"

jnguiot commented 4 years ago

Hello,

I have recently encounter this strange behavior on our tests. I have boiled it down to an issue with a module named dkimpy. However, I don't understand what is the issue and why an import would crash pylint. It seems to be linked to the new version of pylint

Even if I am experiencing some issues with pylint today, I would like to thank you for this wonderful and useful piece of software.

Steps to reproduce

Install latest pylint astroid : pip install pylint astroid --pre -U
Install dkimpy: pip install pylint
Create test.py :
```
import dkim
```
Pylint it: pylint test.py

Notes

test.py can be executed without issue (of course, it outputs nothing)
It was working in 2.4.4

Current behavior

It crashes with this error:

************* Module test
test.py:1:0: C0114: Missing module docstring (missing-module-docstring)
Traceback (most recent call last):
  File "/home/jng/test/aaa/bin/pylint", line 8, in <module>
    sys.exit(run_pylint())
  File "/home/jng/test/aaa/lib/python3.7/site-packages/pylint/__init__.py", line 22, in run_pylint
    PylintRun(sys.argv[1:])
  File "/home/jng/test/aaa/lib/python3.7/site-packages/pylint/lint/run.py", line 338, in __init__
    linter.check(args)
  File "/home/jng/test/aaa/lib/python3.7/site-packages/pylint/lint/pylinter.py", line 871, in check
    self.get_ast, self._iterate_file_descrs(files_or_modules)
  File "/home/jng/test/aaa/lib/python3.7/site-packages/pylint/lint/pylinter.py", line 904, in _check_files
    self._check_file(get_ast, check_astroid_module, name, filepath, modname)
  File "/home/jng/test/aaa/lib/python3.7/site-packages/pylint/lint/pylinter.py", line 930, in _check_file
    check_astroid_module(ast_node)
  File "/home/jng/test/aaa/lib/python3.7/site-packages/pylint/lint/pylinter.py", line 1063, in check_astroid_module
    ast_node, walker, rawcheckers, tokencheckers
  File "/home/jng/test/aaa/lib/python3.7/site-packages/pylint/lint/pylinter.py", line 1107, in _check_astroid_module
    walker.walk(ast_node)
  File "/home/jng/test/aaa/lib/python3.7/site-packages/pylint/utils/ast_walker.py", line 75, in walk
    self.walk(child)
  File "/home/jng/test/aaa/lib/python3.7/site-packages/pylint/utils/ast_walker.py", line 72, in walk
    callback(astroid)
  File "/home/jng/test/aaa/lib/python3.7/site-packages/pylint/checkers/variables.py", line 1177, in visit_import
    module = next(_infer_name_module(node, parts[0]))
  File "/home/jng/test/aaa/lib/python3.7/site-packages/astroid/util.py", line 160, in limit_inference
    yield from islice(iterator, size)
  File "/home/jng/test/aaa/lib/python3.7/site-packages/astroid/context.py", line 113, in cache_generator
    for result in generator:
  File "/home/jng/test/aaa/lib/python3.7/site-packages/astroid/decorators.py", line 132, in raise_if_nothing_inferred
    yield next(generator)
  File "/home/jng/test/aaa/lib/python3.7/site-packages/astroid/decorators.py", line 96, in wrapped
    res = next(generator)
  File "/home/jng/test/aaa/lib/python3.7/site-packages/astroid/inference.py", line 254, in infer_import
    yield self.do_import_module(name)
  File "/home/jng/test/aaa/lib/python3.7/site-packages/astroid/mixins.py", line 100, in do_import_module
    modname, level=level, relative_only=level and level >= 1
  File "/home/jng/test/aaa/lib/python3.7/site-packages/astroid/scoped_nodes.py", line 642, in import_module
    return MANAGER.ast_from_module_name(absmodname)
  File "/home/jng/test/aaa/lib/python3.7/site-packages/astroid/manager.py", line 189, in ast_from_module_name
    return self.ast_from_file(found_spec.location, modname, fallback=False)
  File "/home/jng/test/aaa/lib/python3.7/site-packages/astroid/manager.py", line 98, in ast_from_file
    return AstroidBuilder(self).file_build(filepath, modname)
  File "/home/jng/test/aaa/lib/python3.7/site-packages/astroid/builder.py", line 137, in file_build
    module = self._data_build(data, modname, path)
  File "/home/jng/test/aaa/lib/python3.7/site-packages/astroid/builder.py", line 190, in _data_build
    module = builder.visit_module(node, modname, node_file, package)
  File "/home/jng/test/aaa/lib/python3.7/site-packages/astroid/rebuilder.py", line 165, in visit_module
    newnode.postinit([self.visit(child, newnode) for child in node.body])
  File "/home/jng/test/aaa/lib/python3.7/site-packages/astroid/rebuilder.py", line 165, in <listcomp>
    newnode.postinit([self.visit(child, newnode) for child in node.body])
  File "/home/jng/test/aaa/lib/python3.7/site-packages/astroid/rebuilder.py", line 177, in visit
    return visit_method(node, parent)
  File "/home/jng/test/aaa/lib/python3.7/site-packages/astroid/rebuilder.py", line 676, in visit_functiondef
    return self._visit_functiondef(nodes.FunctionDef, node, parent)
  File "/home/jng/test/aaa/lib/python3.7/site-packages/astroid/rebuilder.py", line 666, in _visit_functiondef
    body=[self.visit(child, newnode) for child in node.body],
  File "/home/jng/test/aaa/lib/python3.7/site-packages/astroid/rebuilder.py", line 666, in <listcomp>
    body=[self.visit(child, newnode) for child in node.body],
  File "/home/jng/test/aaa/lib/python3.7/site-packages/astroid/rebuilder.py", line 177, in visit
    return visit_method(node, parent)
  File "/home/jng/test/aaa/lib/python3.7/site-packages/astroid/rebuilder.py", line 911, in visit_return
    newnode.postinit(self.visit(node.value, newnode))
  File "/home/jng/test/aaa/lib/python3.7/site-packages/astroid/rebuilder.py", line 177, in visit
    return visit_method(node, parent)
  File "/home/jng/test/aaa/lib/python3.7/site-packages/astroid/rebuilder.py", line 384, in visit_binop
    self._bin_op_classes[type(node.op)], node.lineno, node.col_offset, parent
KeyError: <class '_ast.Sub'>

Expected behavior

It should not crash. If there is a problem with import, it should deliver an understandable message.

pylint --version output

pylint 2.5.0
astroid 2.4.0
Python 3.7.3 (default, Dec 20 2019, 18:57:59)
[GCC 8.3.0]

PCManticore commented 4 years ago

Thanks for the report! This should already be fixed in the 2.4 and master branches on astroid. Please give it a go and let us know if you encounter any issues. We'll release the fix as astroid 2.4.1 in a couple of days.

jnguiot commented 4 years ago

Indeed I can confirm it works with the latest 2.4 version of astroid. Thank you for the quick fix!

pylint-dev / pylint