python / mypy

Optional static typing for Python
https://www.mypy-lang.org/
Other
18.33k stars 2.81k forks source link

`Enum` created via functional API should be iterable #16936

Open abravalheri opened 7 months ago

abravalheri commented 7 months ago

Bug Report

The Python docs website document enum items as iterables:

>>> Animal = Enum('Animal', 'ANT BEE CAT DOG')
>>> Animal
<enum 'Animal'>
>>> Animal.ANT
<Animal.ANT: 1>
>>> list(Animal)
[<Animal.ANT: 1>, <Animal.BEE: 2>, <Animal.CAT: 3>, <Animal.DOG: 4>]

https://docs.python.org/3/howto/enum.html#functional-api

This definition is also used in typeshed: https://github.com/python/typeshed/blob/a2095002e446bd0e20f8a469a14e271cd6cc6ad9/stdlib/enum.pyi#L105.

However, mypy fails to recognise this characteristic when using the functional API.

To Reproduce

# Gist URL: https://gist.github.com/mypy-play/d41bcc40b2df9dbfb6f5a0cff7c18681
# Playground URL: https://mypy-play.net/?mypy=latest&python=3.12&gist=d41bcc40b2df9dbfb6f5a0cff7c18681
from enum import Enum

(LATEST,) = Enum("v", "LATEST")

DOWNLOADS = {
    "file1.txt": LATEST,
    "file2.html": 7,
    "file3.css": 3,
    "file4.txt": 5,
    "file5.txt": LATEST,
}

def download_all() -> None:
    for file, version in DOWNLOADS.items():
        if version is LATEST:
            url = f"http://latest.example.url/files/{file}"
        else:
            url = f"http://example.url/files/version/{version}/{file}"
        _download(url)

def _download(url: str) -> None:
    print(f"... pretend to be downloading {url} ...")

if __name__ == "__main__":
    download_all()

Note that I am using Enum in this example because it is much nicer to work with than LATEST = object(), it has a very good __repr__ and if at any point I need to add more special values, I can do that easily. Overall Enum's functional API is an all-rounder for defining constants.

Expected Behavior

Mypy should find no error when checking the example.

Indeed no runtime error can be found, and the program runs as expected:

$ python3.11 main.py
... pretend to be downloading http://latest.example.url/files/file1.txt ...
... pretend to be downloading http://example.url/files/version/7/file2.html ...
... pretend to be downloading http://example.url/files/version/3/file3.css ...
... pretend to be downloading http://example.url/files/version/5/file4.txt ...
... pretend to be downloading http://latest.example.url/files/file5.txt ...

Actual Behavior

main.py:3: error: "Enum" object is not iterable  [misc]
main.py:6: error: Cannot determine type of "LATEST"  [has-type]
main.py:10: error: Cannot determine type of "LATEST"  [has-type]
main.py:15: error: Cannot determine type of "LATEST"  [has-type]
Found 4 errors in 1 file (checked 1 source file)

Your Environment


A related problem (but not identical use case) has been previously closed in the tracker #8009. Please note, however, that the explanations (^1, ^2) for closing that other issue do not apply to this use case:

In summary, the other issue described a situation where an Enum was constantly re-defined as a class attribute inside of class' __init__ function. The report was dismissed because the constant re-definition of the Enum would cause subtle bugs.

In this use case however, the Enum is only defined once during the evaluation of the module by the import machinery. It is never re-evaluated and the Enum "class" itself is not assigned as a member of another object.

erictraut commented 7 months ago

Mypy is able to handle this pattern if you make the following small modification to your code.

v = Enum("v", "LATEST")
LATEST = v.LATEST

This approach is preferable from a static type checking perspective because it allows the type checker to evaluate a more specific (narrower) type for LATEST: Literal[v.LATEST]. By contrast, if you rely on the iterable interface to Enum, the evaluated type will be v.

abravalheri commented 7 months ago

Hi @erictraut, thank you very much for the suggestion.

Yes, this is a very good workaround for the bug in the question.

The unnecessary verboseness (and unnecessary introduction of an extra binding in the module namespace) are not really my cup of tea, so for the time being, I will just add an ignore comment until hopefully the problem goes away in some decades down the line 🤞.

Anyway, the error message is very misleading.