search

pylint-dev / pylint

It's not just a linter that annoys you!
https://pylint.readthedocs.io/en/latest/
GNU General Public License v2.0
5.31k stars 1.13k forks source link

Latest update 2.5.0 broke --extension-pkg-whitelist #3542

Closed evandrocoan closed 4 years ago

evandrocoan commented 4 years ago

Steps to reproduce

After reverting our pylint to use astroid 2.3.3, everything started working again: https://github.com/ankitects/pylint/pull/1 - Fix astroid update breaking build

Anyways, the problem can be reproduced with the latest pylint and astroid versions.

Our builds were working fine until come out the latest update of astroid to 2.4.0: https://github.com/ankitects/anki/runs/624211313#step:27:1153

python -m pylint -j 0 --rcfile=.pylintrc -f colorized  \
   --extension-pkg-whitelist=PyQt5,ankirspy aqt tests setup.py
multiprocessing.pool.RemoteTraceback: 
************* Module aqt
aqt/__init__.py:1:0: F0002: <class 'RuntimeError'>: wrapped C/C++ object of type QApplication has been deleted (astroid-error)
"""
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.7.6/x64/lib/python3.7/multiprocessing/pool.py", line 121, in worker
    result = (True, func(*args, **kwds))
  File "/home/runner/work/anki/anki/pyenv/lib/python3.7/site-packages/pylint/lint.py", line 1342, in _worker_check_single_file
    _worker_linter.check_single_file(name, filepath, modname)
  File "/home/runner/work/anki/anki/pyenv/lib/python3.7/site-packages/pylint/lint.py", line 995, in check_single_file
    self.get_ast, check_astroid_module, name, filepath, modname
  File "/home/runner/work/anki/anki/pyenv/lib/python3.7/site-packages/pylint/lint.py", line 1035, in _check_file
    check_astroid_module(ast_node)
  File "/home/runner/work/anki/anki/pyenv/lib/python3.7/site-packages/pylint/lint.py", line 1168, in check_astroid_module
    ast_node, walker, rawcheckers, tokencheckers
  File "/home/runner/work/anki/anki/pyenv/lib/python3.7/site-packages/pylint/lint.py", line 1212, in _check_astroid_module
    walker.walk(ast_node)
  File "/home/runner/work/anki/anki/pyenv/lib/python3.7/site-packages/pylint/utils/ast_walker.py", line 77, in walk
    self.walk(child)
  File "/home/runner/work/anki/anki/pyenv/lib/python3.7/site-packages/pylint/utils/ast_walker.py", line 74, in walk
    callback(astroid)
  File "/home/runner/work/anki/anki/pyenv/lib/python3.7/site-packages/pylint/checkers/variables.py", line 1156, in visit_import
    module = next(_infer_name_module(node, parts[0]))
  File "/home/runner/work/anki/anki/pyenv/lib/python3.7/site-packages/astroid/util.py", line 160, in limit_inference
    yield from islice(iterator, size)
  File "/home/runner/work/anki/anki/pyenv/lib/python3.7/site-packages/astroid/context.py", line 113, in cache_generator
    for result in generator:
  File "/home/runner/work/anki/anki/pyenv/lib/python3.7/site-packages/astroid/decorators.py", line 132, in raise_if_nothing_inferred
    yield next(generator)
  File "/home/runner/work/anki/anki/pyenv/lib/python3.7/site-packages/astroid/decorators.py", line 96, in wrapped
    res = next(generator)
  File "/home/runner/work/anki/anki/pyenv/lib/python3.7/site-packages/astroid/inference.py", line 254, in infer_import
    yield self.do_import_module(name)
  File "/home/runner/work/anki/anki/pyenv/lib/python3.7/site-packages/astroid/mixins.py", line 100, in do_import_module
    modname, level=level, relative_only=level and level >= 1
  File "/home/runner/work/anki/anki/pyenv/lib/python3.7/site-packages/astroid/scoped_nodes.py", line 642, in import_module
    return MANAGER.ast_from_module_name(absmodname)
  File "/home/runner/work/anki/anki/pyenv/lib/python3.7/site-packages/astroid/manager.py", line 189, in ast_from_module_name
    return self.ast_from_file(found_spec.location, modname, fallback=False)
  File "/home/runner/work/anki/anki/pyenv/lib/python3.7/site-packages/astroid/manager.py", line 98, in ast_from_file
    return AstroidBuilder(self).file_build(filepath, modname)
  File "/home/runner/work/anki/anki/pyenv/lib/python3.7/site-packages/astroid/builder.py", line 138, in file_build
    return self._post_build(module, encoding)
  File "/home/runner/work/anki/anki/pyenv/lib/python3.7/site-packages/astroid/builder.py", line 155, in _post_build
    self.add_from_names_to_locals(from_node)
  File "/home/runner/work/anki/anki/pyenv/lib/python3.7/site-packages/astroid/builder.py", line 208, in add_from_names_to_locals
    imported = node.do_import_module()
  File "/home/runner/work/anki/anki/pyenv/lib/python3.7/site-packages/astroid/mixins.py", line 100, in do_import_module
    modname, level=level, relative_only=level and level >= 1
  File "/home/runner/work/anki/anki/pyenv/lib/python3.7/site-packages/astroid/scoped_nodes.py", line 642, in import_module
    return MANAGER.ast_from_module_name(absmodname)
  File "/home/runner/work/anki/anki/pyenv/lib/python3.7/site-packages/astroid/manager.py", line 189, in ast_from_module_name
    return self.ast_from_file(found_spec.location, modname, fallback=False)
  File "/home/runner/work/anki/anki/pyenv/lib/python3.7/site-packages/astroid/manager.py", line 98, in ast_from_file
    return AstroidBuilder(self).file_build(filepath, modname)
  File "/home/runner/work/anki/anki/pyenv/lib/python3.7/site-packages/astroid/builder.py", line 138, in file_build
    return self._post_build(module, encoding)
  File "/home/runner/work/anki/anki/pyenv/lib/python3.7/site-packages/astroid/builder.py", line 155, in _post_build
    self.add_from_names_to_locals(from_node)
  File "/home/runner/work/anki/anki/pyenv/lib/python3.7/site-packages/astroid/builder.py", line 208, in add_from_names_to_locals
    imported = node.do_import_module()
  File "/home/runner/work/anki/anki/pyenv/lib/python3.7/site-packages/astroid/mixins.py", line 100, in do_import_module
    modname, level=level, relative_only=level and level >= 1
  File "/home/runner/work/anki/anki/pyenv/lib/python3.7/site-packages/astroid/scoped_nodes.py", line 642, in import_module
    return MANAGER.ast_from_module_name(absmodname)
  File "/home/runner/work/anki/anki/pyenv/lib/python3.7/site-packages/astroid/manager.py", line 168, in ast_from_module_name
    return self.ast_from_module(module, modname)
  File "/home/runner/work/anki/anki/pyenv/lib/python3.7/site-packages/astroid/manager.py", line 262, in ast_from_module
    return AstroidBuilder(self).module_build(module, modname)
  File "/home/runner/work/anki/anki/pyenv/lib/python3.7/site-packages/astroid/builder.py", line 95, in module_build
    node = self.inspect_build(module, modname=modname, path=path)
  File "/home/runner/work/anki/anki/pyenv/lib/python3.7/site-packages/astroid/raw_building.py", line 308, in inspect_build
    self.object_build(node, module)
  File "/home/runner/work/anki/anki/pyenv/lib/python3.7/site-packages/astroid/raw_building.py", line 360, in object_build
    elif hasattr(member, "__all__"):
RuntimeError: wrapped C/C++ object of type QApplication has been deleted
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.7.6/x64/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/opt/hostedtoolcache/Python/3.7.6/x64/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/runner/work/anki/anki/pyenv/lib/python3.7/site-packages/pylint/__main__.py", line 7, in <module>
    pylint.run_pylint()
  File "/home/runner/work/anki/anki/pyenv/lib/python3.7/site-packages/pylint/__init__.py", line 23, in run_pylint
    PylintRun(sys.argv[1:])
  File "/home/runner/work/anki/anki/pyenv/lib/python3.7/site-packages/pylint/lint.py", line 1769, in __init__
    linter.check(args)
  File "/home/runner/work/anki/anki/pyenv/lib/python3.7/site-packages/pylint/lint.py", line 983, in check
    self, self.config.jobs, self._iterate_file_descrs(files_or_modules)
  File "/home/runner/work/anki/anki/pyenv/lib/python3.7/site-packages/pylint/lint.py", line 1305, in check_parallel
    _worker_check_single_file, files
  File "/opt/hostedtoolcache/Python/3.7.6/x64/lib/python3.7/multiprocessing/pool.py", line 748, in next
    raise value
RuntimeError: wrapped C/C++ object of type QApplication has been deleted
make[1]: *** [.build/lint] Error 1
Makefile:110: recipe for target '.build/lint' failed
make[1]: Leaving directory '/home/runner/work/anki/anki/qt'
make: *** [check] Error 2
Makefile:156: recipe for target 'check' failed

After minimizing the problem, I reduce it to the following minimal code which is working fine before the recent updates:

  1. pip install pylint astroid --pre -U
  2. Create a file named test.py with the contents: from PyQt5 import QtWidgets
  3. Now run python -m pylint --extension-pkg-whitelist=PyQt5 test.py 
    D:\User\Downloads>python -m pylint --extension-pkg-whitelist=PyQt5 test.py
************* Module test
test.py:1:0: C0114: Missing module docstring (missing-module-docstring)
Traceback (most recent call last):
File "F:\Python\lib\runpy.py", line 193, in _run_module_as_main
return _run_code(code, main_globals, None,
File "F:\Python\lib\runpy.py", line 86, in _run_code
exec(code, run_globals)
File "F:\Python\lib\site-packages\pylint\__main__.py", line 18, in <module>
pylint.run_pylint()
File "F:\Python\lib\site-packages\pylint\__init__.py", line 22, in run_pylint
PylintRun(sys.argv[1:])
File "F:\Python\lib\site-packages\pylint\lint\run.py", line 338, in __init__
linter.check(args)
File "F:\Python\lib\site-packages\pylint\lint\pylinter.py", line 870, in check
self._check_files(
File "F:\Python\lib\site-packages\pylint\lint\pylinter.py", line 904, in _check_files
self._check_file(get_ast, check_astroid_module, name, filepath, modname)
File "F:\Python\lib\site-packages\pylint\lint\pylinter.py", line 930, in _check_file
check_astroid_module(ast_node)
File "F:\Python\lib\site-packages\pylint\lint\pylinter.py", line 1062, in check_astroid_module
retval = self._check_astroid_module(
File "F:\Python\lib\site-packages\pylint\lint\pylinter.py", line 1107, in _check_astroid_module
walker.walk(ast_node)
File "F:\Python\lib\site-packages\pylint\utils\ast_walker.py", line 75, in walk
self.walk(child)
File "F:\Python\lib\site-packages\pylint\utils\ast_walker.py", line 72, in walk
callback(astroid)
File "F:\Python\lib\site-packages\pylint\checkers\variables.py", line 1203, in visit_importfrom
self._check_module_attrs(node, module, name.split("."))
File "F:\Python\lib\site-packages\pylint\checkers\variables.py", line 1868, in _check_module_attrs
module = next(module.getattr(name)[0].infer())
File "F:\Python\lib\site-packages\astroid\scoped_nodes.py", line 547, in getattr
result = [self.import_module(name, relative_only=True)]
File "F:\Python\lib\site-packages\astroid\scoped_nodes.py", line 642, in import_module
return MANAGER.ast_from_module_name(absmodname)
File "F:\Python\lib\site-packages\astroid\manager.py", line 168, in ast_from_module_name
return self.ast_from_module(module, modname)
File "F:\Python\lib\site-packages\astroid\manager.py", line 262, in ast_from_module
return AstroidBuilder(self).module_build(module, modname)
File "F:\Python\lib\site-packages\astroid\builder.py", line 95, in module_build
node = self.inspect_build(module, modname=modname, path=path)
File "F:\Python\lib\site-packages\astroid\raw_building.py", line 308, in inspect_build
self.object_build(node, module)
File "F:\Python\lib\site-packages\astroid\raw_building.py", line 360, in object_build
elif hasattr(member, "__all__"):
RuntimeError: wrapped C/C++ object of type QApplication has been deleted

pylint --version output

D:\User\Downloads>pylint --version
pylint 2.5.0
astroid 2.4.0
Python 3.8.1 (default, Jan  8 2020, 15:55:49) [MSC v.1916 64 bit (AMD64)]
PCManticore commented 4 years ago

This has been fixed in https://github.com/PyCQA/astroid/commit/5e7aed79bf8104f53849b9126cb3ec1329634ce4 which will be part of astroid 2.4.1 in a couple of days (still going through the recent issues to see if there are any crashes we can quickly fix)