Function returning union of type variables has inference/unification behaviour change when assigned to explicitly-typed location

[huonw]

Bug Report

It appears that if a function returns a union of type variables, how mypy chooses to assign/unify those type variables changes if a call happens in a location that has an explicit type (e.g. assigned to a variable with an annotation or in a direct return f(...) call within a function that has a return type hint).

In particular, it appears both type variables end up being unified to that explicit hint, rather than to individual components of the hint's union (for instance, in the reproducer below, it seems that it chooses _T1 = _T2 = int | str, but "should" be _T1 = int and _T2 = str).

This is observable when those parameters are used in contravariant position in an argument, such as the parameters of functions, and can result in some seemingly spurious errors.

(I tried searching for existing discussion/bugs and didn't find any, but I may not have found the exactly correct terms.)

To Reproduce

Reduced example (i.e. not relying on typeshed's hints): https://mypy-play.net/?mypy=latest&python=3.12&gist=697d00f7b3a089d7e56f6d86ec5193e8

from typing import TypeVar, Callable

def expects_int(x: int) -> None:
    pass

def expects_str(x: str) -> None:
    pass

_T1 = TypeVar("_T1")
_T2 = TypeVar("_T2")

def f(key: Callable[[_T1], object], default: Callable[[_T2], object]) -> _T1 | _T2:
    raise NotImplementedError()

# error: Argument 1 to "f" has incompatible type "Callable[[int], None]"; expected "Callable[[int | str], object]"  [arg-type]
# error: Argument 2 to "f" has incompatible type "Callable[[str], None]"; expected "Callable[[int | str], object]"  [arg-type]
broken: int | str = f(expects_int, expects_str)

# no error, but `works` has type `int | str` too
works = f(expects_int, expects_str)
# note: Revealed type is "Union[builtins.int, builtins.str]"
reveal_type(works)

This happens in practice with functions like min with the interaction between the key functions and default: https://mypy-play.net/?mypy=latest&python=3.12&gist=7f892a21b5d3d1db89419620e3cedf47

def broken(x: list[int]) -> int | None:
    # error: Unsupported operand types for + ("None" and "int")  [operator]
    # note: Left operand is of type "int | None"
    return min(x, key=lambda x: x + 1, default=None)

def works(x: list[int]) -> int | None:
    result = min(x, key=lambda x: x + 1, default=None)
    return result

Expected Behavior

Both the broken and works code should behave the same, and in particular, they should both work.

Actual Behavior

main.py:18: error: Argument 1 to "f" has incompatible type "Callable[[int], None]"; expected "Callable[[int | str], object]"  [arg-type]
main.py:18: error: Argument 2 to "f" has incompatible type "Callable[[str], None]"; expected "Callable[[int | str], object]"  [arg-type]
main.py:23: note: Revealed type is "Union[builtins.int, builtins.str]"

Your Environment

• Mypy version used: 1.7.1
• Mypy command-line flags: n/a (see mypy-play.net)
• Mypy configuration options from mypy.ini (and other config files): n/a
• Python version used: 3.12

[allisonkarlitskaya]

Here's another case which I think is caused by the same underlying issue:

from typing import TypeVar

DT = TypeVar('DT')

def getstr(default: DT | None = None) -> str | DT:
    assert False

# That's fine
reveal_type(getstr())  # Revealed type is "builtins.str"

z = None
# Unexpectedly broken
reveal_type(z or getstr())  # Revealed type is "Union[builtins.str, None]"

# But oddly, that way works fine
reveal_type(z if z else getstr())  # Revealed type is "builtins.str"

observed with mypy 1.8.0 (compiled: no)