Irregularities in populating new dict from a source dict with a narrower key type

[macdjord]

Bug Report

When creating or updating a new dict with items from another dict with a narrower key type, Mypy complains about some methods which are actually perfectly valid.

To Reproduce

from typing import TypeVar

_KT = TypeVar("_KT")
_VT = TypeVar("_VT")

def clone_dict(source_dict: dict[_KT, _VT]) -> dict[_KT, _VT]:
    return dict(source_dict)

def test(source_dict: dict[str, None]) -> list[dict[str | int, None]]:
    a: dict[str | int, None] = {"a": None, "b": None}
    b: dict[str | int, None] = {k: v for k, v in source_dict.items()}
    c: dict[str | int, None] = {k: None for k in source_dict.keys()}
    d: dict[str | int, None] = {**source_dict}
    e: dict[str | int, None] = dict(source_dict.items())
    f: dict[str | int, None] = dict(source_dict)
    g: dict[str | int, None] = source_dict.copy()
    h: dict[str | int, None] = clone_dict(source_dict)
    i: dict[str | int, None] = {}
    i.update(source_dict.items())
    j: dict[str | int, None] = {}
    j.update(source_dict)

    return [a, b, c, d, e, f, g, h, i, j]

Expected Behavior

The above code shows 10 ways of creating the new dictionary:

• a doesn't actually clone the source dict; it just creates a new dict from a dict literal that just happens to only contain string keys
• b copies the source dict using a dict comprehension and .items()
• c uses a dict comprehension and .keys() (possible in this case because the dict value type is None, which is a singleton)
• d uses a dict comprehension and star-star-expansion
• e uses the dict() constructor and .items()
• f uses the dict() constructor and the source dict directly
• g uses the built-in .copy() method
• h uses a user-created dict copying method
• i creates an empty dict and then .update()s it using .items()
• j creates an empty dict and then .update()s it using the source dict directly

Of these:

• Mypy should complain about h; Mypy has no way of knowing that the function in question is returning a fresh dict which is used nowhere else (and thus can safely be implicitly cast to a wider type)
• Mypy might complain about g; since that is a built-in method, it could be special-cased into Mypy that this is safe, but absent such a special case it would have the same semantics as clone_dict()
• All the other option are safe and should not raise any error

Actual Behavior

Mypy correctly warns about g and h, but also issues spurious warnings about d, f, and j:

$ mypy test.py
test.py:15: error: Unpacked dict entry 0 has incompatible type "dict[str, None]"; expected "SupportsKeysAndGetItem[str | int, None]"  [dict-item]
test.py:17: error: Incompatible types in assignment (expression has type "dict[str, None]", variable has type "dict[str | int, None]")  [assignment]
test.py:18: error: Incompatible types in assignment (expression has type "dict[str, None]", variable has type "dict[str | int, None]")  [assignment]
test.py:19: error: Argument 1 to "clone_dict" has incompatible type "dict[str, None]"; expected "dict[str | int, None]"  [arg-type]
test.py:23: error: Argument 1 to "update" of "MutableMapping" has incompatible type "dict[str, None]"; expected "SupportsKeysAndGetItem[str | int, None]"  [arg-type]
Found 5 errors in 1 file (checked 1 source file)

Your Environment

• Mypy version used: mypy 1.5.1 (compiled: yes)
• Mypy command-line flags: None
• Mypy configuration options from mypy.ini (and other config files): None
• Python version used: Python 3.11.2

[erictraut]

I agree with your analysis except for j, which should generate an error based on the typeshed definition of the dict.update method.

For comparison, pyright generates errors for g, h and j.

[macdjord]

@erictraut: i and j are functionally identical; I see no reason why one should cause a warning when the other does not.

[erictraut]

@macdjord, from a typing perspective, i and j are different. With i, you're passing a value of type dict_items[str, None]. With j, you're passing a value of type dict[str, None]. The dict.update method accepts a value of type Iterable[tuple[str | int, None]]. The type dict_items[str, None] is compatible with this type, but dict[str, None] is not.

from typing import Iterable
from _collections_abc import dict_items

def func(d_i: dict_items[str, None], d_j: dict[str, None]):
    v: Iterable[tuple[str | int, None]]

    v = d_i  # No type error
    v = d_j  # Type error

[macdjord]

Well, yes, obviously v = d_j would be a type error, because dict[KT, VT] is Iterable[KT], not Iterable[tuple[KT, VT]]. But that's not actually a problem in my example because dict.update() accepts Iterable[tuple[KT, VT]] or Mapping[KT, VT].

[erictraut]

... because dict.update() accepts Iterable[tuple[KT, VT]] or Mapping[KT, VT]

Actually, it doesn't accept Mapping[KT, VT], but it does accept SupportsKeysAndGetItem[KT, VT]. Regardless, both Mapping and SupportsKeysAndGetItem define the KT type parameter as invariant, which means str is not compatible with str | int.

By contrast, the type parameters for Iterable and tuple are covariant, which is why I was focusing on the Iterable[tuple[KT, VT]] overload.

def func(d_j: dict[str, None]):
    s: SupportsKeysAndGetItem[str | int, None]
    s = d_j  # Type error

    m: Mapping[str | int, None]
    m = d_j  # Type error