python / mypy

Optional static typing for Python
https://www.mypy-lang.org/
Other
18.49k stars 2.83k forks source link

False positive - Type narrowing of class attributes #17537

Open Tinche opened 3 months ago

Tinche commented 3 months ago

I'm not super sure if this is a bug or not, but the behavior is different in Mypy and pyright so probably worth discussing it.

To Reproduce

https://mypy-play.net/?mypy=latest&python=3.12&gist=3a34812b38b40ca5cb913ee48daf2c49

from dataclasses import dataclass
from enum import Enum

from typing_extensions import reveal_type

class E(Enum):
    ONE: int = 1
    TWO: int = 2

@dataclass
class Test:
    a: E

    def set_a(self, value: E) -> None:
        self.a = value

a = Test(E.ONE)
reveal_type(a.a)
assert a.a == E.ONE
reveal_type(a.a)
a.set_a(E.TWO)
reveal_type(a.a)

I ran into this while trying to enable strict_equality on a codebase.

The problem is the last reveal_type call - Mypy claims the type is still Literal[E.ONE], but it's obviously Literal[E.TWO] (or at least E). If strict_equality is enabled, this will cause a type error when asserting in a test, for example.

Pyright doesn't seem to narrow here, claiming the type of a.a is E always.

I don't know what the correct answer is here. Maybe allow narrowing only on local variables? 🤷

JukkaL commented 1 month ago

Yeah, this can be a problem. I'm tagging this as feature, since mypy is working "as designed", though here the behavior is not what is expected.

The narrowing behavior has been around for a very long time, but I think the situation got worse when mypy started narrowing enums on assignment. One option would be to experiment with less eagerly narrowing enums to literal types on attribute assignments/assertions/comparisons. Disabling narrowing of attribute access more widely may cause a lot of new errors, so we need to careful there.

Dreamsorcerer commented 2 weeks ago

Just to add another example (from aiohttp codebase):

        if self._at_eof:
            return b""
        data = bytearray()
        while not self._at_eof:
            data.extend(await self.read_chunk(self.chunk_size))
        if decode:  # unreachable error here
            return self.decode(data)
        return data

Mypy thinks the code is unreachable because it has narrowed the attribute to False and thinks the loop will never exit.

Dreamsorcerer commented 2 weeks ago

I don't know how complex it'd be to implement, but I'm think the best approach could be to do the type narrowing, but anytime a method is called on that object, or an await happens to yield to the event loop, then the type narrowing should be reset.

So:

assert foo.bar is True
foo.bar  # narrowed to True
foo.something()
foo.bar  # bool again