python / mypy

Optional static typing for Python
https://www.mypy-lang.org/
Other
18.51k stars 2.83k forks source link

Allow method assignment in body of type extending typing.NamedTuple #8543

Open asottile opened 4 years ago

asottile commented 4 years ago

this currently works for non-namedtuple classes (both runtime and mypy-time):

class C:
    x: int

    __hash__ = object.__hash__

and it works for typing.NamedTuple at runtime:

from typing import NamedTuple

class C(NamedTuple):
    x: int

    __hash__ = object.__hash__
$ python3 -i t2.py
>>> c = C(1)
>>> hash(c)
-9223363269847169460
>>> object.__hash__(c)
-9223363269847169460
>>> tuple.__hash__(c)
3430019387558

but fails at type-checking time:

$ mypy --version && python --version --version
mypy 0.770
Python 3.6.8 (default, Oct  7 2019, 12:59:55) 
[GCC 8.3.0]
$ mypy t2.py
t2.py:7: error: NamedTuple field name cannot start with an underscore: __hash__
t2.py:7: error: Invalid statement in NamedTuple definition; expected "field_name: field_type [= default]"
Found 2 errors in 1 file (checked 1 source file)

concrete usecase: __hash__ is in the critical path (~13%) for some code I'm writing and the extra layer of function which calls into object.__hash__'s overhead is significant:

# mypy is ok with this
from typing import NamedTuple

class C(NamedTuple):
    x: int

    def __hash__(self) -> int:
        return object.__hash__(self)

I'd be happy to hack on this given some pointers :)

ethanhs commented 4 years ago

I think we could take a PR to fix this, but I'm not entirely sure what the ground truth of allowable dunder overrides are for namedtuple. Perhaps we can allowlist those.

JelleZijlstra commented 4 years ago

I don't think dunders are the issue. At runtime, only objects with type annotations are interpreted as NamedTuple fields, so mypy should mirror that logic.

JukkaL commented 4 years ago

The relevant code is in mypy.semanal_namedtuple.

I agree that it would be better to match runtime semantics here, but it seems fairly low priority. A simple PR that fixes this would probably be fine.

Akuli commented 3 years ago

Easy workaround:

from typing import NamedTuple, TYPE_CHECKING
class C(NamedTuple):
    x: int
    if not TYPE_CHECKING:
        __hash__ = object.__hash__
asottile commented 3 years ago

that doesn't help, typed code which calls those methods will fail

AlexWaygood commented 2 years ago

that doesn't help, typed code which calls those methods will fail

Can't you just do this instead?

from typing import NamedTuple, TYPE_CHECKING
class C(NamedTuple):
    x: int
    if TYPE_CHECKING:
        def __hash__(self) -> int: ...
    else:
        __hash__ = object.__hash__
asottile commented 2 years ago

that also doesn't help because the code outside of if TYPE_CHECKING is not checked -- __hash__ is just a minimal example here -- for example utilizing a shared method definition being assigned to many namedtuple classes

similar to:

class HasParticularAttr(Protocol):
   ...

def f(self: HasParticularAttr) -> int:
    return ...

class C(NamedTuple):
    x: attr

    f = f

class D(NamedTuple):
    x: attr

    f = f
AlexWaygood commented 2 years ago

To be clear, I agree that it would be great if mypy could support this feature. I was just trying to offer an alternative workaround in the meantime.