search

pylint-dev / pylint

It's not just a linter that annoys you!
https://pylint.readthedocs.io/en/latest/
GNU General Public License v2.0
5.23k stars 1.12k forks source link

pylint crashed with a ``AstroidError`` #8866

Closed kornoa closed 2 months ago

kornoa commented 1 year ago

Bug description

pylint crashed with a AstroidError

Configuration

[MESSAGES CONTROL]
disable=
    import-error,
    invalid-name,
    missing-module-docstring,
    missing-class-docstring,
    missing-function-docstring,
    too-few-public-methods,
    too-many-branches,
    too-many-locals,

Command used

python -m pylint pytemplate.py

Pylint output

************* Module pytemplate
pytemplate.py:5:0: C0103: Class name "__metaclass__" doesn't conform to PascalCase naming style (invalid-name)
pytemplate.py:13:0: C0115: Missing class docstring (missing-class-docstring)
pytemplate.py:18:4: C0116: Missing function or method docstring (missing-function-docstring)
pytemplate.py:23:4: C0116: Missing function or method docstring (missing-function-docstring)
Exception on node <Name.__module__ l.26 at 0x7f7fe5af1330> in file '/home/knoack/git-projects/tbs/ansible_collections/controlware/tbs/utils/pytemplate.py'
Traceback (most recent call last):
  File "/home/knoack/git-projects/tbs/.venv/lib/python3.10/site-packages/pylint/utils/ast_walker.py", line 91, in walk
    callback(astroid)
  File "/home/knoack/git-projects/tbs/.venv/lib/python3.10/site-packages/pylint/checkers/variables.py", line 1573, in visit_name
    self._undefined_and_used_before_checker(node, stmt)
  File "/home/knoack/git-projects/tbs/.venv/lib/python3.10/site-packages/pylint/checkers/variables.py", line 1614, in _undefined_and_used_before_checker
    action, nodes_to_consume = self._check_consumer(
  File "/home/knoack/git-projects/tbs/.venv/lib/python3.10/site-packages/pylint/checkers/variables.py", line 1772, in _check_consumer
    ) = self._is_variable_violation(
  File "/home/knoack/git-projects/tbs/.venv/lib/python3.10/site-packages/pylint/checkers/variables.py", line 2129, in _is_variable_violation
    and defframe.locals[node.name][0].lineno < frame.lineno
TypeError: '<' not supported between instances of 'NoneType' and 'int'
pytemplate.py:1:0: F0002: pytemplate.py: Fatal error while checking 'pytemplate.py'. Please open an issue in our bug tracker so we address this. There is a pre-filled template that you can use in '/home/knoack/.cache/pylint/pylint-crash-2023-07-20-13-04-04.txt'. (astroid-error)

------------------------------------------------------------------
Your code has been rated at 0.00/10 (previous run: 0.00/10, +0.00)

Expected behavior

No exception

Pylint version

python -m pylint --version
pylint 2.17.4
astroid 2.15.6
Python 3.10.6 (main, May 29 2023, 11:10:38) [GCC 11.3.0]

OS / Environment

cat /etc/os-release PRETTY_NAME="Ubuntu 22.04.2 LTS" NAME="Ubuntu" VERSION_ID="22.04" VERSION="22.04.2 LTS (Jammy Jellyfish)" VERSION_CODENAME=jammy ID=ubuntu ID_LIKE=debian HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" UBUNTU_CODENAME=jammy

Additional dependencies

cat /home/knoack/.cache/pylint/pylint-crash-2023-07-20-13-04-04.txt First, please verify that the bug is not already filled: https://github.com/PyCQA/pylint/issues/

Then create a new crash issue: https://github.com/PyCQA/pylint/issues/new?assignees=&labels=crash%2Cneeds+triage&template=BUG-REPORT.yml

Issue title: Crash ```` (if possible, be more specific about what made pylint crash) Content: When parsing the following file:

"""Class to run and handle custom template using python code"""

from __future__ import absolute_import, division, print_function

__metaclass__ = type

import os
import sys
import importlib.util
from ansible.plugins.action import AnsibleActionFail

class PyTemplate:
    def __init__(self) -> None:
        self.loaded_modules = {}
        self.template_path = ""

    def run_template(self, template_vars: any) -> str:
        if template_vars is not None:
            return ""
        return ""

    def set_template_path(self, path: str) -> None:
        self.template_path = path

    def load_module_from_file(self, name: str, path: str) -> __module__:
        """
        name: system wide name of module (sys["modules"])
        path: relative path to pyhton code file (with or without .py ending)
        """
        if name in self.loaded_modules:
            return self.loaded_modules[name]

        py_ending = ".py"
        stripped_path = path.rstrip(py_ending)
        module_path = os.path.join(self.template_path, stripped_path + py_ending)
        if not os.path.isfile(module_path):
            raise FileExistsError(f"Cannot find module '{module_path}'.")
        try:
            spec = importlib.util.spec_from_file_location(name, module_path)
            imported_module = importlib.util.module_from_spec(spec)
            sys.modules[name] = imported_module
            self.loaded_modules[name] = imported_module
            spec.loader.exec_module(imported_module)
        except ImportError as ie:
            raise AnsibleActionFail(
                f"Cannot import python template {module_path}."
            ) from ie
        return imported_module

pylint crashed with a AstroidError and with the following stacktrace:

Traceback (most recent call last):
  File "/home/knoack/git-projects/tbs/.venv/lib/python3.10/site-packages/pylint/lint/pylinter.py", line 811, in _lint_file
    check_astroid_module(module)
  File "/home/knoack/git-projects/tbs/.venv/lib/python3.10/site-packages/pylint/lint/pylinter.py", line 1085, in check_astroid_module
    retval = self._check_astroid_module(
  File "/home/knoack/git-projects/tbs/.venv/lib/python3.10/site-packages/pylint/lint/pylinter.py", line 1135, in _check_astroid_module
    walker.walk(node)
  File "/home/knoack/git-projects/tbs/.venv/lib/python3.10/site-packages/pylint/utils/ast_walker.py", line 94, in walk
    self.walk(child)
  File "/home/knoack/git-projects/tbs/.venv/lib/python3.10/site-packages/pylint/utils/ast_walker.py", line 94, in walk
    self.walk(child)
  File "/home/knoack/git-projects/tbs/.venv/lib/python3.10/site-packages/pylint/utils/ast_walker.py", line 94, in walk
    self.walk(child)
  File "/home/knoack/git-projects/tbs/.venv/lib/python3.10/site-packages/pylint/utils/ast_walker.py", line 91, in walk
    callback(astroid)
  File "/home/knoack/git-projects/tbs/.venv/lib/python3.10/site-packages/pylint/checkers/variables.py", line 1573, in visit_name
    self._undefined_and_used_before_checker(node, stmt)
  File "/home/knoack/git-projects/tbs/.venv/lib/python3.10/site-packages/pylint/checkers/variables.py", line 1614, in _undefined_and_used_before_checker
    action, nodes_to_consume = self._check_consumer(
  File "/home/knoack/git-projects/tbs/.venv/lib/python3.10/site-packages/pylint/checkers/variables.py", line 1772, in _check_consumer
    ) = self._is_variable_violation(
  File "/home/knoack/git-projects/tbs/.venv/lib/python3.10/site-packages/pylint/checkers/variables.py", line 2129, in _is_variable_violation
    and defframe.locals[node.name][0].lineno < frame.lineno
TypeError: '<' not supported between instances of 'NoneType' and 'int'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/knoack/git-projects/tbs/.venv/lib/python3.10/site-packages/pylint/lint/pylinter.py", line 775, in _lint_files
    self._lint_file(fileitem, module, check_astroid_module)
  File "/home/knoack/git-projects/tbs/.venv/lib/python3.10/site-packages/pylint/lint/pylinter.py", line 813, in _lint_file
    raise astroid.AstroidError from e
astroid.exceptions.AstroidError
mbyrnepr2 commented 1 year ago

Thanks for this @kornoa! The reproducer can be simplified to:

class A:
    def say_hello(self) -> __module__:
        ...
mbyrnepr2 commented 1 year ago

At first glance, defframe.locals[node.name][0] is an astroid node representing the name of the module in which PyTemplate is defined and which doesn't have a lineno.

anordal commented 11 months ago

I got another NoneType in an Astroid callback, but with this code:

from dataclasses import dataclass
from enum import Enum

@dataclass
class Status(Enum):
    OFF = 0
Exception on node <AssignName.OFF l.7 at 0x7f11f4f0c1d0> in file '/home/an/git/zivid-sdk/relay_control.py'
Traceback (most recent call last):
  File "venv/lib64/python3.11/site-packages/pylint/utils/ast_walker.py", line 91, in walk
    callback(astroid)
  File "venv/lib64/python3.11/site-packages/pylint/checkers/base/name_checker/checker.py", line 488, in visit_assignname
    if utils.is_enum_member(node) or utils.is_assign_name_annotated_with(
       ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "venv/lib64/python3.11/site-packages/pylint/checkers/utils.py", line 2284, in is_enum_member
    enum_member_objects = frame.locals.get("__members__")[0].items
                          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^
TypeError: 'NoneType' object is not subscriptable

I can reproduce it in a minimal venv from this requirements.txt:

#
# Generated by:
# echo pylint > requiremetns.in && pip-compile --strip-extras requirements.in
#
astroid==3.0.0
dill==0.3.7
isort==5.12.0
mccabe==0.7.0
platformdirs==3.11.0
pylint==3.0.0
tomlkit==0.12.1

From this, I can also reproduce the simplified reproducer above.

DanielNoord commented 11 months ago

Thanks for checking old issues. That seems to be https://github.com/pylint-dev/pylint/issues/9100 which is resolved on master!