python / mypy

Optional static typing for Python
https://www.mypy-lang.org/
Other
18.55k stars 2.84k forks source link

Cannot set return type of asynccontextmanagers in Protocol classes #17996

Open tilboerner opened 1 month ago

tilboerner commented 1 month ago

Bug Report

When decorating an empty method ("...") inside of a Protocol definition as an asynccontextmanager, the type checker insists that the return type must use Never instead of allowing to specify the type intended by the protocol.

To Reproduce

See also in Playground.

from collections.abc import AsyncIterator
from contextlib import asynccontextmanager
from typing import Protocol

class P(Protocol):
    @asynccontextmanager
    async def f(self) -> AsyncIterator[None]:
        """Create a context but yield nothing."""
        ...

Importing AsyncIterator from typing instead of collections.abc makes no difference.

Expected Behavior

I should be allowed to annotate the expected yield-Type of the context manager in a Protocol without having to provide an implementation. (None in this case, but also any other type that the context will yield to the with statement.)

Actual Behavior

Produces this error:

main.py:7: error: Argument 1 to "asynccontextmanager" has incompatible type "Callable[[P], Coroutine[Any, Any, AsyncIterator[None]]]"; expected "Callable[[P], AsyncIterator[Never]]"  [arg-type]
Found 1 error in 1 file (checked 1 source file)

Your Environment

My mypy config (but it also reproduces when leaving all options unchecked in the Playground):

[tool.mypy]
ignore_missing_imports = false
disallow_untyped_calls = true
disallow_untyped_defs = true
disallow_incomplete_defs = true
disallow_any_unimported = true
strict_optional = true
no_implicit_optional = true
check_untyped_defs = true
show_error_codes = true
warn_unused_configs = true
warn_redundant_casts = true
warn_unused_ignores = true
warn_return_any = true
warn_unreachable = true
strict_equality = true
strict = true
tilboerner commented 1 month ago

Workaround:

provide an implementation in the Protocol Definition that yield an instance of the correct type:

from collections.abc import AsyncIterator
from contextlib import asynccontextmanager
from typing import Protocol

class P(Protocol):
    @asynccontextmanager
    async def f(self) -> AsyncIterator[None]: 
        """Create a context but yield nothing."""
        yield None  # Workaround for https://github.com/python/mypy/issues/17996
JelleZijlstra commented 1 month ago

You should use a def instead of an async def. Due to the way async generators are detected by CPython, an async def without a yield has a different type.

This unfortunate edge case comes up pretty regularly; we should add it to https://mypy.readthedocs.io/en/stable/common_issues.html.

tilboerner commented 1 month ago

@JelleZijlstra Thanks, using a def without async in the Protocol method definition works indeed, even when concrete implementations use async def.

It's pretty surprising, having to bend the Prototype definition away from what will get implemented, and in effect making the Prototype less representative of what is expected.

If the error is expected to stay, then having it mentioned in Common Issues, as you suggest, is a good idea. Not only is it the natural place to find such information; it's also a respectable and high-ranking source that will hopefully improve web search results.

To help with the latter, I suggest also including the error message in the common issues document.

hauntsaninja commented 1 month ago

There's reasonable docs for this in https://mypy.readthedocs.io/en/stable/more_types.html#typing-async-await

A PR that mentions the error message in "Common issues" and links to the above section would be welcome