python / mypy

Optional static typing for Python
https://www.mypy-lang.org/
Other
18.13k stars 2.76k forks source link

type inferance from await asyncio.gather() with 6 elements no longer works #17030

Open jacob-keller opened 5 months ago

jacob-keller commented 5 months ago

Bug Report

mypy doesn't seem to properly infer types when unpacking the results of an asyncio.gather() that has at least 6 elements.

This leads to the unpacked variables getting assigned the object type, and then future uses of the variable cause typing issues.

To Reproduce

import asyncio
import datetime

async def get_str_value() -> str:
    await asyncio.sleep(1)

    return 'value'

async def get_datetime_value() -> datetime.datetime:
    await asyncio.sleep(2)

    return datetime.datetime.now()

async def run() -> None:

    (str1_value,
     str2_value,
     str3_value,
     datetime1_value,
     str4_value,
     str5_value,
     datetime2_value) = reveal_type(await asyncio.gather(get_str_value(),
                                                         get_str_value(),
                                                         get_str_value(),
                                                         get_datetime_value(),
                                                         get_str_value(),
                                                         get_str_value(),
                                                         get_datetime_value()))

    reveal_type(str1_value)
    reveal_type(datetime1_value)

    print(f"str={str1_value}, date={datetime1_value}")

asyncio.run(run())

Expected Behavior

mypy should be able to infer the appropriate type of the values from the gathered list. From searching, it looks like this might be a typeshed issue, or it might be something in mypy itself. It is unclear what can be done here.

Actual Behavior

mypy infers the return of the await as a list, and then assigns the object type to each of the variables. This results in cascading type errors as other code assumes those variables would be the appropriate type.

If you only have 5 values, then this correctly unpacks the tuple into the specific variables. But as soon as you gather 6 items it breaks.

asyncio-gather-test.py:21: note: Revealed type is "tuple[builtins.str, builtins.str, builtins.str, datetime.datetime, builtins.str, builtins.str]" asyncio-gather-test.py:29: note: Revealed type is "builtins.str" asyncio-gather-test.py:30: note: Revealed type is "datetime.datetime"

Your Environment

I'm really not sure what a good workaround in my own code would be here. For now I've taken to using asserts to check the types, which works ok, but is verbose.

erictraut commented 5 months ago

Mypy's behavior is correct here. The typeshed definition for asyncio.gather provides overloads for one to six arguments. You're passing seven arguments, so the behavior is dictated by a fallback overload:

    def gather(*coros_or_futures: _FutureLike[_T], return_exceptions: Literal[False] = False) -> Future[list[_T]]: ...

Because mypy uses joins rather than unions, the resulting return type is Future[list[object]]. (By contrast, pyright evaluates the return type as Future[list[str | datetime]]).

To support seven or more arguments, the typeshed definition would need to be extended to include more overloads.

jacob-keller commented 5 months ago

Right that makes sense. I suspect this used to work with an older version of typeshed at some point. I'll investigate there if it's worth fixing or if I can add my own overload to my project or something