Wrong typing warning for map. (Argument X to "map" has incompatible type overloaded function)

[y9c]

I write this function to check the rightmost common element of two list.

this is the code.

import itertools

def get_common_index(*names: str) -> int:
    """zip genalogy history name tag."""

    return max(
        i
        for i, x in enumerate(
            map(
                lambda x: len(set(x)) > 1,
                list(reversed(list(itertools.zip_longest(*map(reversed, names), fillvalue="0")))),
            )
        )
        if x
    )

print(get_common_index("1110111111", "101101111")

It works fine, and the input types are correct.

$ python test.py

5

But mypy raise the following error message.

$ mypy test.py

test.py:24: error: Argument 1 to "map" has incompatible type overloaded function; expected "Callable[[str], Iterator[_T]]"

Is this an error of mypy?

[JelleZijlstra]

I can reproduce this on master. Mypy is also extremely slow to typecheck this fairly small file:

LOG:  Processing SCC singleton (gci) as inherently stale
gci.py:11: error: Argument 1 to "map" has incompatible type overloaded function; expected "Callable[[str], Iterator[_T]]"
LOG:  Deleting gci gci.py gci.meta.json gci.data.json
LOG:  No fresh SCCs left in queue
LOG:  Build finished in 200.799 seconds with 20 modules, and 96 errors

Maybe it hits some pathological case because there are so many nested functions with complex types?

As for the originally reported bug, it also repros with this simpler code:

def get_common_index(*names: str) -> int:
    x = map(reversed, names)
    return 0

Here is another repro that doesn't rely on the typeshed types of map and reversed:

from typing import Iterator, Any, TypeVar, overload, List, Tuple, Callable, Iterable, Sequence

_T = TypeVar("_T")
_S = TypeVar("_S")

@overload
def reversed2(obj: Tuple[_T, ...]) -> Iterator[_T]:
    pass

@overload
def reversed2(obj: Sequence[_T]) -> Iterator[_T]:
    pass

def reversed2(obj: Any) -> Any:
    raise Exception

def map2(func: Callable[[_T], _S], iter1: Iterable[_T]) -> Iterator[_S]:
    raise Exception

def get_common_index(names: List[str]) -> None:
    map2(reversed2, names)

produces gci.py:26: error: Argument 1 to "map2" has incompatible type overloaded function; expected "Callable[[str], Iterator[_T]]" (on master)

[ilevkivskyi]

It may be just another manifestation of https://github.com/python/mypy/issues/1317: passing one generic function to another generic function doesn't work correctly.

[ilevkivskyi]

Also I am worried about the performance here, 200 seconds for few lines of not so deeply nested code is too much.

[JelleZijlstra]

I think you're right, the previous example in #3193 is very similar. Maybe we should keep this issue open to deal with the performance issue.

[ilevkivskyi]

Maybe we should keep this issue open to deal with the performance issue.

Good idea! Should we add a label for performance issues?

[ilevkivskyi]

OK, I added the label and applied to a bunch of issues.

[AlexWaygood]

The performance issues with this snippet of code seem to still be here on mypy 0.942, though I can't reproduce anything as severe as 200 seconds.

The error message for the original example is now different (I think due to changes in typeshed):

def get_common_index(*names: str) -> int:
    x = map(reversed, names)  # error: Argument 1 to "map" has incompatible type "Type[reversed[Any]]"; expected "Callable[[str], reversed[_T]]"
    return 0

@JelleZijlstra's repro that avoids the typeshed stubs still reproduces exactly, however.