Typing spec change for `Annotated` special form

[erictraut]

I'd like to request that the TC consider adoption of change to the typing spec that clarifies type checking behaviors for Annotated when it is used in a value expression.

Links to PR & Discussion The PR can be found here.

The discussion can be found here.

TC Sign-off

• [x] Eric
• [x] Guido
• [x] Jelle
• [x] Rebecca
• [x] Shantanu

Current Type Checker Behaviors Mypy and pyre already reject the use of Annotated when assigned to type[T]. The latest version of pyright (1.1.350) does likewise, making it consistent with mypy and pyre and the proposed spec clarification.

The latest version of pyright (1.1.350) also rejects the use of Annotated (or a type alias thereof) in a call expression. The other type checkers do not yet enforce this.

Controversial Issues Several people in the thread argued for more permissive behaviors here, but the general consensus was in favor of the proposed clarification.

Several people also argued that type checkers should wait to implement the clarified behaviors until after TypeForm has been specified. I think this decision should be left up to individual type checker maintainers.

[JelleZijlstra]

Mypy and pyre already reject the use of Annotated when assigned to type[T].

Mypy does not reject this when the Annotated form is first assigned to a variable (I mentioned this in https://discuss.python.org/t/is-annotated-compatible-with-type-t/43898/43). I don't think that's a principled behavior, however.

[rchen152]

I'm fine with the proposed spec change, but I do want to point out that it contradicts some text in the typing docs and the remainder of the spec about treating Annotated as its underlying type:

If a library or tool encounters an annotation Annotated[T, x] and has no special logic for the metadata, it should ignore the metadata and simply treat the annotation as T. As such, Annotated can be useful for code that wants to use annotations for purposes outside Python’s static typing system.

Using Annotated[T, x] as an annotation still allows for static typechecking of T, as type checkers will simply ignore the metadata x.

(https://docs.python.org/3/library/typing.html#typing.Annotated)

When a tool or a library does not support annotations or encounters an unknown annotation it should just ignore it and treat annotated type as the underlying type.

(https://typing.readthedocs.io/en/latest/spec/qualifiers.html#consuming-annotations)

If we're all in agreement that type checkers should instead treat Annotated as a special form, then we should probably update the rest of the documentation to reflect this.

[erictraut]

The documentation clearly uses the term "annotation", which underscores the point that Annotated was designed to be used in type annotations, and its behaviors are defined for that usage. The original PEP and official documentation are silent on what it means to use Annotated in value expressions (outside of annotations). It seems that such a usage wasn't contemplated by the original PEP authors. The proposed change attempts to clarify this.

[rchen152]

I agree that this is a good clarification. What I'm concerned about is how it interacts with the claim that type checkers can just treat Annotated[T, x] as T in type annotations.

Here's an example:

MyInt: TypeAlias = Annotated[int, "metadata"]
my_type: type[Any] = MyInt

If I can treat Annotated[int, "metadata"] like int, I would expect to be able to simplify the first line to MyInt: TypeAlias = int. But that'll cause the second line to pass type-checking, whereas the spec will now say that the second line ought to produce an error.

At least in the spec, since type checker maintainers are one of the intended audiences, I think we should make it clear that Annotated cannot just be ignored. I guess the user-facing typing documentation might be fine as-is, although I still think it's a little misleading.

[gvanrossum]

I'm signing off on this clarification. While, like Rebecca and some posters on the thread, I'm a tiny little bit concerned with backwards compatibility, but in the end I agree that we can't expect Annotated[int, ""] to behave exactly as int in all circumstances, and we might as well clarify that it isn't a type and cannot be called to instantiate something. People manipulating objects created through Annotated[...] at runtime should not declare variables holding such values as type -- they just aren't.

[rchen152]

Fair enough. I'll sign off on this change, too, since my objection amounts to wanting other parts of the typing documentation clarified, which can be done later.