1.11 regression: "generic" mapping failing with confusing error message

[asottile]

Bug Report

I've boiled down a minimal example -- this pattern is used by pyupgrade (and reorder-python-imports) to register ast functions and has been working with mypy since version 0.710 -- but was broken with the last update sadly

To Reproduce

this is a simplified version of pyupgrade/_data.py -- FUNCS acts as a mapping from ast types to their callback functions

import ast
import collections
from typing import Callable, Protocol, TypeVar

AST_T = TypeVar('AST_T', bound=ast.AST)
ASTFunc = Callable[[AST_T], None]

FUNCS = collections.defaultdict(list)

def register(tp: type[AST_T]) -> Callable[[ASTFunc[AST_T]], ASTFunc[AST_T]]:
    def register_decorator(func: ASTFunc[AST_T]) -> ASTFunc[AST_T]:
        FUNCS[tp].append(func)
        return func
    return register_decorator

class ASTCallbackMapping(Protocol):
    def __getitem__(self, tp: type[AST_T]) -> list[ASTFunc[AST_T]]: ...

def visit(funcs: ASTCallbackMapping) -> None: ...

def f() -> None:
    visit(FUNCS)

Expected Behavior

the behaviour pre-mypy-1.11:

(no errors)

Actual Behavior

$ mypy t.py 
t.py:24: error: Argument 1 to "visit" has incompatible type "defaultdict[type[AST_T], list[Callable[[AST_T], None]]]"; expected "ASTCallbackMapping"  [arg-type]
t.py:24: note: Following member(s) of "defaultdict[type[AST_T], list[Callable[[AST_T], None]]]" have conflicts:
t.py:24: note:     Expected:
t.py:24: note:         def [AST_T: AST] __getitem__(self, type[AST_T], /) -> list[Callable[[AST_T], None]]
t.py:24: note:     Got:
t.py:24: note:         def __getitem__(self, type[AST_T], /) -> list[Callable[[AST_T], None]]
Found 1 error in 1 file (checked 1 source file)

the error message seems suspicious because I think those two are the same ?

Your Environment

• Mypy version used: 1.11.0
• Mypy command-line flags: none
• Mypy configuration options from mypy.ini (and other config files): none
• Python version used: 3.10.12 -- though really anything modern

[hauntsaninja]

Bisects to https://github.com/python/mypy/pull/17311 , looks like some weird partial type situation where mypy gets a partial type from the FUNCS[tp].append but the type var scope there is different than the scope in ASTCallbackMapping.__getitem__.

Definitely should at least have a better error message, a possible workaround is FUNCS = cast(ASTCallbackMapping, collections.defaultdict(list))

[erictraut]

Another workaround is to make AST_T a class-scoped type variable in the protocol definition:

class ASTCallbackMapping(Protocol[AST_T]):
    def __getitem__(self, tp: type[AST_T]) -> list[ASTFunc[AST_T]]: ...

[asottile]

making the Protocol generic isn't right either because then visit can't be typed properly -- the actual __getitem__ is what's generic, not the class itself