python / mypy

Optional static typing for Python
https://www.mypy-lang.org/
Other
18.19k stars 2.78k forks source link

mypy cannot determine Enum members from static or Final values #17187

Open straz opened 4 months ago

straz commented 4 months ago

Bug Report To Reproduce

Here are 3 variations on creating IntEnum or StrEnum using functional API syntax.

testing1.py: (Success)

UpperSeat = IntEnum("UpperSeat", {"Seat1": 1, "Seat2": 2, "Seat3": 3})

testing2.py: (Error)

UPPER_CODES: Final[dict[str, int]] = {"Seat1": 1, "Seat2": 2, "Seat3": 3}
UpperSeat = IntEnum("UpperSeat", UPPER_CODES)

testing3.py: (Error)

UPPER_CODES: Final[dict[str, int]] = {f"Seat{code}": code for code in range(1, 3)}
UpperSeat = IntEnum("UpperSeat", UPPER_CODES)

Expected Behavior

All three of these produce valid Python code, but only one of these passes mypy type analysis. I would argue that UPPER_CODES in both testing2 and testing3 are static. In testing2.py, UPPER_CODES is a static constant. In testing3.py, UPPER_CODES is provably static. In both testing2 and testing3, Final values are used, so mypy should not be throwing this error.

Actual Behavior

testing1.py - Uses static values in-line: mypy yields Success testing2.py and testing3.py: mypy yields Error:

main.py:8: error: Second argument of IntEnum() must be string, tuple, list or dict literal for mypy to determine Enum members  [misc]
main.py:9: error: Second argument of StrEnum() must be string, tuple, list or dict literal for mypy to determine Enum members  [misc]
Found 2 errors in 1 file (checked 1 source file)

Your Environment

kreathon commented 4 months ago

Note that Final only means that the variable should not be reassigned (see here) and not that the dict is immutable.

So the following code is actually fine:

UPPER_CODES: Final[dict[str, int]] = {"Seat1": 1, "Seat2": 2, "Seat3": 3}
UPPER_CODES["Seat4"] = 4
UpperSeat = IntEnum("UpperSeat", UPPER_CODES)

... and mutable dictionaries make it really difficult do get static code analysis right.

You could, of course, argue that some kind of immutable dictionary wrapper / constructs like MappingProxyType should then be valid inputs for the IntEnum , but again it gets really dynamic making it difficult for static code analysis.

All three of these produce valid Python code, but only one of these passes mypy type analysis.

In the scenario here, mypy needs to find a balance between making use of IntEnum created with the functional syntax, and producing false-positives for some ("valid") dynamic Python programs.

Also note that it is totally okay that you can write Python programs that will not "pass" mypy (so it is not necessarily a "bug").

Trivial example:

x = 3
x = "3"  #  error: Incompatible types in assignment (expression has type "str", variable has type "int")  [assignment]
straz commented 4 months ago

I appreciate what you're saying, but static code analysis of my three examples would show that UPPER_CODES is not modified before its value is used to define UpperSeat. The value of UpperSeat is guaranteed to be fully known to mypy in this example, so I feel that static code analysis would be appropriate here.

In your example, the line UPPER_CODES["Seat4"] = 4 would of course mutate the value, but in my examples there is no such line. More importantly, there is no code that could be added to another file which could possibly inject such a change. I continue to claim that static code analysis should be able to guarantee values of UpperSeat.

kreathon commented 4 months ago

I also cannot think of any way how to break it (with no other statement in between).

Initially, I understood the issue as a "more general problem" instead of this specific example (the Final is actually not of interest here).

So I guess you are right ;)

(Note that I am not a mypy developer, I was just randomly reading it.)