python / mypy

Optional static typing for Python
https://www.mypy-lang.org/
Other
18.61k stars 2.85k forks source link

Enum member aliases does not work correctly with Literal #8657

Open kprzybyla opened 4 years ago

kprzybyla commented 4 years ago

I'm not entirely sure whether this is a bug or missing feature related to the literals and enums but I think it is worth noticing.

Let's say that we have an enum and we define aliases for enum member outside of the enum's scope:

import enum

from typing import Literal

class Color(enum.Enum):

    BLACK = enum.auto()

BLACK = Color.BLACK
BLACK_ALIAS: Literal[Color.BLACK] = Color.BLACK

reveal_type(Color.BLACK)
reveal_type(BLACK)
reveal_type(BLACK_ALIAS)

Running mypy on above code results in following output:

example.py:12: note: Revealed type is 'Literal[Color.BLACK]?'
example.py:13: note: Revealed type is 'Color'
example.py:14: note: Revealed type is 'Literal[Color.BLACK]'

Now it's not idea that second reveal does not reveal Color.BLACK but it's also not that much of a problem since we can add Literal[Color.BLACK typehint and the see the expected result in third reveal. The bigger problem comes, when we want to use Literal on such aliases:

import enum

from typing import Literal, Optional

class Color(enum.Enum):

    BLACK = enum.auto()

BLACK = Color.BLACK
BLACK_ALIAS: Literal[Color.BLACK] = Color.BLACK

x: Optional[Literal[Color.BLACK]] = None
y: Optional[Literal[BLACK]] = None
z: Optional[Literal[BLACK_ALIAS]] = None

reveal_type(x)
reveal_type(y)
reveal_type(z)

Running mypy on above code results in following output:

example.py:16: note: Revealed type is 'Union[Literal[Color.BLACK], None]'
example.py:17: note: Revealed type is 'Union[Any, None]'
example.py:18: note: Revealed type is 'Union[Any, None]'

It's clear that mypy does not interfere those aliases correctly even those the BLACK_ALIAS seemed to be correctly revealed as Literal[Color.BLACK] previously. It would be really nice to have mypy reveal those types correctly.

Now, you might ask, why would you want to define such aliases in the first place? Well, the most probable scenario would be the idea of having a enum class that basically implements a null object pattern. In such cases, you probably don't want to expose the enum class itself to the user.

The most basic example would be this:

import enum

from typing import Literal

class _MissingType(enum.Enum):

    MISSING = enum.auto()

    def __repr__(self) -> str:
        return self.name

MISSING: Literal[_MissingType.MISSING] = _MissingType.MISSING

Now with such construct I don't want to expose _MissingType class but because MISSING alias is not always interfered correctly I cannot do that if I want to have correct typing information everywhere.

I came up with following "reasonable" workaround for the time being:

MissingLiteral = Literal[_MissingType.MISSING]

Which allows me to only expose MissingLiteral and use it in following way:

x: Optional[MissingLiteral] = None

The revealed type is of course Union[Literal[Color.BLACK], None]

ilevkivskyi commented 4 years ago

There is no such thing as enum member aliases, these are just variables (even though possibly with single value). You can use Final however.

kprzybyla commented 4 years ago

There is no such thing as enum member aliases, these are just variables (even though possibly with single value)

Yeah, I couldn't figure out how to call them to describe this behavior and what I need to achieve. Sorry about that :)

The idea of using Final here sound great. Unfortunately it still does not fully resolve the issue unless I'm doing something wrong:

import enum

from typing import Literal, Final, Optional

class Color(enum.Enum):

    BLACK = enum.auto()

BLACK: Final = Color.BLACK
BLACK_ALIAS: Final[Literal[Color.BLACK]] = Color.BLACK

# This is now ok
reveal_type(Color.BLACK)  # Revealed type is 'Literal[scratch_56.Color.BLACK]?'
reveal_type(BLACK)        # Revealed type is 'Literal[scratch_56.Color.BLACK]?'
reveal_type(BLACK_ALIAS)  # Revealed type is 'Literal[scratch_56.Color.BLACK]'

x: Optional[Literal[Color.BLACK]] = None
y: Optional[Literal[BLACK]] = None
z: Optional[Literal[BLACK_ALIAS]] = None

# This however still does not work
reveal_type(x)  # Revealed type is 'Union[Literal[scratch_56.Color.BLACK], None]'
reveal_type(y)  # Revealed type is 'Union[Any, None]'
reveal_type(z)  # Revealed type is 'Union[Any, None]'

Type annotation for y and z is still not what I'm expecting.

I guess that this is caused by the fact that Literal only accepts certain values and enum member value assigned to variable defined outside of enum scope is not seen as in the same way as enum member inside the enum scope even when used with the Final annotation.

And this is actually reasonable in the sense that above example with enums is basically the same as:

from typing import Literal, Final, Optional

black: Literal[5] = 5
BLACK: Final[Literal[5]] = black

x: Optional[Literal[BLACK]] = None

reveal_type(black)  # Revealed type is 'Literal[5]'
reveal_type(BLACK)  # Revealed type is 'Literal[5]'
reveal_type(x)      # Revealed type is 'Union[Any, None]'

And usage of something like Literal[BLACK] is illegal as stated in the mypy documentation. So Literal probably treats my case as arbitrary expression because of that. Now I'm not fully sure whether this behavior that I'm expecting is the correct one or not.

ilevkivskyi commented 4 years ago

Hm, there may be an even simpler solution if you want to use it in annotations: just create a type alias:

BLACK = Literal[Color.BLACK]

x: Optional[BLACK]
reveal_type(x)  # Revealed type is 'Union[Literal[main.Color.BLACK], None]'

Then however it will not be valid in runtime context. In mypy you can't really (and shouldn't) mix values and types. So you will need to use both:

BLACK_VALUE: Final = Color.BLACK
BLACK_TYPE = Literal[Color.BLACK]

We can probably add some special-casing for enum values, similar to None, so they will be automatically understood as a type or a value depending on context.

I think someone already asked once about this.

kprzybyla commented 4 years ago

Yes, that's exactly what I have in my code base right now as the current solution for this. And this is fine since I only need type annotation in couple places. However with this solution the end-users will unfortunately stumble upon the same issue when adding type annotations in their code unless I explicitly expose those types.

So it's not that big of an issue but it would be really nice if this would work :)

terencehonles commented 3 years ago

I just bumped into this and solved it the same way https://github.com/python/mypy/issues/8657#issuecomment-617585724. I had already done some searching and I had a tab open with this issue while I was basically getting to the same conclusion. I probably should have just read this because it is exactly the case I was trying to solve and I may have been able to save a little time just reading the conclusion.

It looks like I followed basically the same path as the OP and it would be nice if the type alias did not need to be created alongside the type. It does look like mypy is being defensive about the value passed to Literal and it does not "realize" the value being passed is a Literal already (but reveal_type does indeed recognize it as a Literal).

Anyways, that's just a long way to say +1 :sweat_smile:

noah-built commented 2 years ago

agreed, would love to fix this!