pylint-dev / astroid

A common base representation of python source code for pylint and other projects
https://pylint.readthedocs.io/projects/astroid/en/latest/
GNU Lesser General Public License v2.1
519 stars 268 forks source link

`Enum`: Mismatch between `__members__` and the members infered by `astroid` #1730

Open Bibo-Joshi opened 1 year ago

Bibo-Joshi commented 1 year ago

Steps to reproduce

Here is an MWE that show the mismatch:

from enum import Enum

from astroid import builder

class MyEnum(Enum):
    REGULAR_MEMBER = 1
    _PROTECTED_MEMBER = 2
    __PRIVATE_MEMBER = 3
    TRAILING_US_MEMBER_ = 5
    # The following should be ignored
    __DUNDER_MEMBER__ = 4
    __members__ = {"custom": "dict"}

instance = builder.extract_node(
    """
    from enum import Enum
    class MyEnum(Enum):
        REGULAR_MEMBER = 1
        _PROTECTED_MEMBER = 2
        __PRIVATE_MEMBER = 3
        TRAILING_US_MEMBER_ = 5
        # The following should be ignored
        __DUNDER_MEMBER__ = 4
        __members__ = {"custom": "dict"}
    """
)

python_members = set(MyEnum.__members__.keys())
astroid_members = {item[1].name for item in instance.getattr("__members__")[0].items}

print("python_members:", python_members)
print("astroid_members:", astroid_members)
print("Wrongly reported members:", astroid_members - python_members)

Current behavior

astroid version 2.12.2 python version 3.10.2

Output of above MWE:


python_members: {'TRAILING_US_MEMBER_', '_MyEnum__PRIVATE_MEMBER', 'REGULAR_MEMBER', '_PROTECTED_MEMBER'}
astroid_members: {'TRAILING_US_MEMBER_', '_PROTECTED_MEMBER', '__DUNDER_MEMBER__', 'REGULAR_MEMBER', '__PRIVATE_MEMBER', '__members__'}
Wrongly reported members: {'__members__', '__PRIVATE_MEMBER', '__DUNDER_MEMBER__'}

Expected behavior

Wrongly reported members should be empty.

The only thing that I'm not entirely sure about is the __PRIVATE_MEMBER. In Py 3.10 I get a

 DeprecationWarning: private variables, such as '_MyEnum__PRIVATE_MEMBER', will be normal attributes in 3.11     
  __PRIVATE_MEMBER = 3

I don't fully understand how private names will be treated in 3.11 (here and here), but in any case astroid should conform with the python behavior IMO.

python -c "from astroid import __pkginfo__; print(__pkginfo__.version)" output

2.12.2

Context

This issue is the result of the discussion at https://github.com/PyCQA/pylint/pull/7257. I managed to track down the enum-logic to https://github.com/PyCQA/astroid/blob/3621e2e7d68653d66cbf770e6dcb61ba541117f1/astroid/brain/brain_namedtuple_enum.py#L358-L460 but unfortunately I currently don't have the resources to dive into this & try to find out how to improve the logic.

DanielNoord commented 1 year ago

I'd like to get #1598 merged before we do any additional work on Enums.