python / cpython

The Python programming language
https://www.python.org
Other
63.49k stars 30.41k forks source link

Calling typing.get_type_hints with a class or instance method with PEP 695 type parameters fails if PEP 563 is enabled #124089

Open Parnassius opened 2 months ago

Parnassius commented 2 months ago

Bug report

Bug description:

from __future__ import annotations

import typing

class Test[M]:
    def foo(self, arg: M) -> None:
        pass

print(typing.get_type_hints(Test.foo))
$ python3.12 pep_695_pep_563.py 
Traceback (most recent call last):
  File "/run/host/tmp/test/pep_695_pep_563.py", line 9, in <module>
    print(typing.get_type_hints(Test.foo))
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/typing.py", line 2310, in get_type_hints
    hints[name] = _eval_type(value, globalns, localns, type_params)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/typing.py", line 415, in _eval_type
    return t._evaluate(globalns, localns, type_params, recursive_guard=recursive_guard)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/typing.py", line 947, in _evaluate
    eval(self.__forward_code__, globalns, localns),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 1, in <module>
NameError: name 'M' is not defined

This seems very similar to #114053, but with class methods (or instance methods, the result is the same) instead of classes themselves.

Removing the from __future__ import annotations import works fine:

import typing

class Test[M]:
    def foo(self, arg: M) -> None:
        pass

print(typing.get_type_hints(Test.foo))
$ python3.12 pep_695_no_563.py 
{'arg': M, 'return': <class 'NoneType'>}

Using the pre PEP-695 syntax, with or without from __future__ import annotations, works fine as well:

from __future__ import annotations

import typing

M = typing.TypeVar("M")

class Test(typing.Generic[M]):
    def foo(self, arg: M) -> None:
        pass

print(typing.get_type_hints(Test.foo))
$ python3.12 no_695_pep_563.py 
{'arg': ~M, 'return': <class 'NoneType'>}
import typing

M = typing.TypeVar("M")

class Test(typing.Generic[M]):
    def foo(self, arg: M) -> None:
        pass

print(typing.get_type_hints(Test.foo))
$ python3.12 no_695_no_563.py 
{'arg': ~M, 'return': <class 'NoneType'>}

This happens on both 3.12.5 and 3.13.0rc2

CPython versions tested on:

3.12, 3.13

Operating systems tested on:

Linux

sobolevn commented 2 months ago

Can replicate it on main:

» ./python.exe ex.py
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython2/ex.py", line 9, in <module>
    print(typing.get_type_hints(Test.foo))
          ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "/Users/sobolev/Desktop/cpython2/Lib/typing.py", line 2465, in get_type_hints
    hints[name] = _eval_type(value, globalns, localns, type_params, format=format, owner=obj)
                  ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sobolev/Desktop/cpython2/Lib/typing.py", line 481, in _eval_type
    return evaluate_forward_ref(t, globals=globalns, locals=localns,
                                type_params=type_params, owner=owner,
                                _recursive_guard=recursive_guard, format=format)
  File "/Users/sobolev/Desktop/cpython2/Lib/typing.py", line 1073, in evaluate_forward_ref
    value = forward_ref.evaluate(globals=globals, locals=locals,
                                 type_params=type_params, owner=owner)
  File "/Users/sobolev/Desktop/cpython2/Lib/annotationlib.py", line 152, in evaluate
    value = eval(code, globals=globals, locals=locals)
  File "<string>", line 1, in <module>
NameError: name 'M' is not defined

I will take a look, thanks for the report.

sobolevn commented 2 months ago

From the first sight - there's not much of what can be done, because Test.foo does not have any refences to Test and has a type <function>. It also does not have any __type_params__ set. And __annotate__ of <function> does not even get called.

There are literally no ways of accessing Test.__type_params__ from Test.foo that I am aware of.

Except for this piece of hackery: obj.__globals__[obj.__qualname__.split('.')[0]].__type_params__ But, I don't think that it is good enough.

I think that we might need some special handling for this.

JelleZijlstra commented 2 months ago

I don't think we should change anything here. from __future__ import annotations does not interact well with runtime introspection, and that's a major part of why in Python 3.14 we're likely to deprecate it (PEP-749).

In the meantime, if you want reliable runtime introspection, don't use from __future__ import annotations.