python / cpython

The Python programming language
https://www.python.org
Other
62.34k stars 29.94k forks source link

`typing.get_type_hints` doesn't support GenericAliases #111353

Open NCPlayz opened 10 months ago

NCPlayz commented 10 months ago

Bug report

Bug description:

Seems like when a GenericAlias is passed into typing.get_type_hints it fails.

from typing import get_type_hints

class Foo[T]:
    type: T

print(get_type_hints(Foo))
print(get_type_hints(Foo[str])) # TypeError: __main__.Foo[str] is not a module, class, method, or function.

I also expected type to resolve to str in this case.

CPython versions tested on:

3.10, 3.11, 3.12

Operating systems tested on:

Linux, macOS

Linked PRs

JelleZijlstra commented 10 months ago

get_type_hints() is documented to "Return a dictionary containing type hints for a function, method, module or class object." A GenericAlias isn't any of these, so this isn't a bug.

We could add support, but it might be better to leave this to external tools that perform runtime type checking.

mikeshardmind commented 10 months ago

This breaks the symmetry between generic and non-generic classes and should probably be supported here. The fact that subscripting a generic class to specify the generic parameters doesn't return a class object is an implementation detail, not "how people use it"

NCPlayz commented 10 months ago

Looks like type-checkers (both pyright and mypy) also act on the basis that a generic alias and class behave the same, and that not having access to e.g. __mro__ is an implementation detail, but type-checker concepts do not always match up with runtime concepts.

Checking the code above also does not fail in mypy or pyright, but there is some concern from @JelleZijlstra that this is because the typeshed stubs allows object, which is why the type-checker may be accepting it (stubs).

I still feel like it makes sense to support GenericAlias for get_type_hints, but as it stands the function has its own bugs as well, when providing a Generic class.

from typing import Generic, get_type_hints

class Foo[T]:
    type: T

class Bar(Foo[str]):
    ...

print(get_type_hints(Foo)) # {'type': ~T}
print(get_type_hints(Bar)) # {'type': ~T}
print(get_type_hints(Foo[str])) # fails

If we wanted to make this (albeit unexpected) behaviour consistent for GenericAliases, seems like it would need something along the lines of this:

        if isinstance(obj, typing._GenericAlias):
            obj = get_origin(obj)

But, to resolve TypeVars all the way down you would need to locate all the TypeVars and substitute them for their values.

This is possible, but it took a lot of effort to figure this one out. I've implemented a utility in this gist for it, but while writing this, I think I may have found an issue with the TypeVar implementation outlined here in a Python discussion topic.

zhPavel commented 10 months ago

I think the standard library needs a more abstract mechanism for resolving generics. For example, the current implementation does not allow inferring the actual types for method parameters.

I can suggest a implementation that I use in my project as a basis. You can find it at: https://github.com/reagento/adaptix/blob/7ca3366aa7edb0b897fe6575c4c68c7e67d17ba0/src/adaptix/_internal/type_tools/generic_resolver.py

It is fully covered by tests: https://github.com/reagento/adaptix/blob/7ca3366aa7edb0b897fe6575c4c68c7e67d17ba0/tests/unit/morphing/model/shape_provider/test_generic_resolving.py

This implementation differs in some aspects from the behavior presented in the linked PR. For example, it parameterizes generics based on the bound and constraints of type variables.

If you start exploring the project, it may seem a bit complex. There are many internal abstractions that are not explicitly described in the documentation, but I am willing to answer any questions you may have. I advise you to avoid looking at normalize_type.py file. It takes responsibility of introspecting type hints and contains a large number of hacks designed to solve the problem of compatibility between different versions. Its purpose is to simplify the analysis of type hints for all higher-level layers of the library. It is used by GenericResolver only to recursively expand Unpack[some_tuple].

I also think it's worth discussing whether get_type_hints should replace Self with type of the parent.