microsoft / pyright

Static Type Checker for Python
Other
13.16k stars 1.41k forks source link

No narrowing happens when using all() in a type guard #9073

Closed geochip closed 2 hours ago

geochip commented 2 hours ago

Describe the bug If a function has several annotated optinal arguments, checking all of them with all([arg1, arg2, ...]) doesn't narrow the type, which results in error diagnostics like

error: Object of type "None" cannot be used as iterable value (reportOptionalIterable)
error: Object of type "None" cannot be called (reportOptionalCall)

Expected behavior: The types of arguments undergo narrowing and error diagnostics are not printed.

Code or Screenshots

from collections.abc import Callable

def foo(
    some_string: str | None = None,
    some_list: list[str] | None = None,
    some_func: Callable[[str], None] | None = None,
):
    if not all([some_string, some_list, some_func]):
        raise ValueError()

    print(some_string[0])
    some_func(some_string)
    for s in some_list:
        print(s)

Running pyright on this file outputs following erros:

error: Object of type "None" is not subscriptable (reportOptionalSubscript)
error: Object of type "None" cannot be called (reportOptionalCall)
error: Argument of type "str | None" cannot be assigned to parameter of type "str"
    Type "str | None" is not assignable to type "str"
      "None" is not assignable to "str" (reportArgumentType)
error: Object of type "None" cannot be used as iterable value (reportOptionalIterable)

If the if statement is replaced with the following:

    if not (some_string and some_func and some_list):
        raise ValueError()

then no errors are printed. The same behavior is expected when using all([...])

Python version: Python 3.12.6

VS Code extension or command-line The issue is present when running pyright as a commad-line tool and as a language server in neovim with command: pyright-langserver --stdio

version: pyright 1.1.381

erictraut commented 2 hours ago

Yes, not all([a, b, c, ...]) is not a supported type guard pattern. For a full list of supported type guard patterns, refer to this documentation.

The following is supported:

    if not some_string or not some_list or not some_func:
        raise ValueError()

Also:

    if not (some_string and some_list and some_func):
        raise ValueError()

We do occasionally add support for new type guard patterns, but each new pattern requires custom logic and can (in some cases significantly) slow down analysis performance. That means the bar is quite high for justifying the addition of new patterns. We'd need to see a strong signal from many pyright users that this pattern is sufficiently common.

I'll note that mypy, another Python type checker, also doesn't implement support for this type narrowing pattern either. There has been an enhancement request filed in the mypy github tracker for this feature, but it hasn't received any thumbs ups. That's a pretty good indication that it's not a commonly-used pattern.

This particular pattern is quite complex since it involves a call expression, a list expression, and multiple expressions within the list expression that could be narrowed. This pattern would also be pretty expensive at analysis time, especially in untyped code bases where pyright needs to infer return types in code based on implementations.

Another issue with this pattern is that it's rare to check for truthiness for a set of variables. It's much more common to check specifically for None. For example, do you really want to raise a ValueError if an empty list is passed to foo?

For more complex type narrowing patterns, you have the option of defining your own user-defined type guard function. For details, refer to the documentation on TypeGuard and TypeIs.

I'm going to reject this enhancement request for now, but it's a decision I might be willing to revisit in the future if there is sufficient signal from other pyright users.