search

python / mypy

Optional static typing for Python
https://www.mypy-lang.org/
Other
17.85k stars 2.74k forks source link

Fix explicit type for partial #17424

Closed ilevkivskyi closed 1 week ago

ilevkivskyi commented 1 week ago

Fixes https://github.com/python/mypy/issues/17301

cc @hauntsaninja

github-actions[bot] commented 1 week ago

Diff from mypy_primer, showing the effect of this PR on open source code:

kopf (https://github.com/nolar/kopf)
- kopf/_core/actions/invocation.py:127: error: Incompatible types in assignment (expression has type "partial[partial[object | Coroutine[None, None, object]]]", variable has type "partial[object | Coroutine[None, None, object | None] | None]")  [assignment]
- kopf/_core/actions/invocation.py:127: error: Argument 1 to "run" of "Context" has incompatible type "partial[object | Coroutine[None, None, object | None] | None]"; expected "Callable[[VarArg(Any), KwArg(Any)], partial[object | Coroutine[None, None, object]]]"  [arg-type]
- kopf/_core/actions/invocation.py:127: note: "partial[object | Coroutine[None, None, object | None] | None].__call__" has type "Callable[[VarArg(Any), KwArg(Any)], object | Coroutine[None, None, object | None] | None]"

jax (https://github.com/google/jax)
+ jax/_src/linear_util.py:367: error: INTERNAL ERROR -- Please try using mypy master on GitHub:
+ https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
+ Please report a bug at https://github.com/python/mypy/issues
+ version: 1.11.0+dev.f628e9aca90e9a8f171d3a3731be07a77ca3bf18
+ jax/_src/linear_util.py:367: : note: use --pdb to drop into pdb
- jax/_src/sharding_specs.py:123: error: Unused "type: ignore" comment  [unused-ignore]
- jax/_src/tree_util.py:769: error: Invalid index type "type[T@_register_keypaths]" for "dict[type[T], _RegistryWithKeypathsEntry]"; expected type "type[T@register_pytree_with_keys]"  [index]
- jax/_src/xla_bridge.py:149: error: Unused "type: ignore" comment  [unused-ignore]
- jax/_src/dtypes.py:347: error: Unused "type: ignore" comment  [unused-ignore]
- jax/_src/dtypes.py:348: error: Unused "type: ignore" comment  [unused-ignore]
- jax/_src/dtypes.py:684: error: Unused "type: ignore" comment  [unused-ignore]
- jax/_src/dtypes.py:741: error: Unused "type: ignore" comment  [unused-ignore]
- jax/_src/linear_util.py:340: error: Missing positional argument "tree" in call to "tree_map"  [call-arg]
- jax/_src/core.py:1463: error: Redundant cast to "AbstractValue"  [redundant-cast]
- jax/_src/api_util.py:345: error: Unused "type: ignore" comment  [unused-ignore]
- jax/_src/interpreters/xla.py:101: error: Unused "type: ignore" comment  [unused-ignore]
- jax/_src/interpreters/xla.py:102: error: Unused "type: ignore" comment  [unused-ignore]
- jax/_src/interpreters/partial_eval.py:2687: error: Argument 2 to "_eval_jaxpr_padded" has incompatible type "Sequence[Any]"; expected "list[Any]"  [arg-type]
- jax/_src/interpreters/mlir.py:94: error: Unused "type: ignore" comment  [unused-ignore]
- jax/_src/interpreters/mlir.py:106: error: Unused "type: ignore" comment  [unused-ignore]
- jax/_src/interpreters/mlir.py:187: error: Unused "type: ignore" comment  [unused-ignore]
- jax/_src/interpreters/mlir.py:272: error: Unused "type: ignore" comment  [unused-ignore]
- jax/_src/interpreters/mlir.py:274: error: Unused "type: ignore" comment  [unused-ignore]
- jax/_src/interpreters/mlir.py:306: error: Unused "type: ignore" comment  [unused-ignore]
- jax/_src/interpreters/mlir.py:310: error: Unused "type: ignore" comment  [unused-ignore]
- jax/_src/interpreters/mlir.py:324: error: Unused "type: ignore" comment  [unused-ignore]
- jax/_src/interpreters/mlir.py:2167: error: Unused "type: ignore" comment  [unused-ignore]
- jax/_src/interpreters/mlir.py:2631: error: Unused "type: ignore" comment  [unused-ignore]
- jax/_src/debugging.py:394: error: Unused "type: ignore" comment  [unused-ignore]
- jax/_src/api.py:1912: error: Unsupported left operand type for != (PyTreeDef?)  [operator]
- jax/_src/api.py:2280: error: Unused "type: ignore" comment  [unused-ignore]
- jax/_src/numpy/util.py:277: error: Unused "type: ignore" comment  [unused-ignore]
- jax/_src/numpy/util.py:286: error: Unused "type: ignore" comment  [unused-ignore]
- jax/_src/lax/other.py:206: error: Redundant cast to "Precision | None"  [redundant-cast]
- jax/_src/interpreters/pxla.py:912: error: Redundant cast to "ShapedArray"  [redundant-cast]
- jax/_src/custom_derivatives.py:247: error: Missing positional argument "tree" in call to "tree_map"  [call-arg]
- jax/_src/lax/lax.py:557: error: Redundant cast to "Array"  [redundant-cast]
- jax/_src/lax/lax.py:622: error: Redundant cast to "Array"  [redundant-cast]
- jax/_src/lax/lax.py:851: error: Redundant cast to "Array"  [redundant-cast]
- jax/_src/lax/lax.py:915: error: Redundant cast to "Array"  [redundant-cast]
- jax/_src/lax/lax.py:1011: error: Redundant cast to "Array"  [redundant-cast]
- jax/_src/lax/lax.py:1044: error: Unsupported left operand type for != (PyTreeDef?)  [operator]
- jax/_src/lax/lax.py:1412: error: Redundant cast to "Array"  [redundant-cast]
- jax/_src/lax/lax.py:1454: error: Unused "type: ignore" comment  [unused-ignore]
- jax/_src/lax/lax.py:1471: error: Unused "type: ignore" comment  [unused-ignore]
- jax/_src/lax/lax.py:4666: error: Unused "type: ignore" comment  [unused-ignore]
- jax/_src/lax/lax.py:5044: error: Unsupported left operand type for != (DType?)  [operator]
- jax/_src/lax/lax.py:5126: error: Redundant cast to "tuple[Precision, Precision]"  [redundant-cast]
- jax/_src/lax/lax.py:5136: error: Redundant cast to "tuple[Precision, Precision]"  [redundant-cast]
- jax/_src/lax/lax.py:5139: error: Redundant cast to "tuple[Precision, Precision]"  [redundant-cast]
- jax/_src/lax/windowed_reductions.py:64: error: Unsupported left operand type for != (PyTreeDef?)  [operator]
- jax/_src/lax/windowed_reductions.py:832: error: Unused "type: ignore" comment  [unused-ignore]
- jax/_src/lax/windowed_reductions.py:907: error: Unused "type: ignore" comment  [unused-ignore]
- jax/_src/lax/control_flow/loops.py:2018: error: Unsupported left operand type for == (DType?)  [operator]
- jax/_src/lax/control_flow/loops.py:2072: error: Unsupported left operand type for != (DType?)  [operator]
- jax/_src/lax/control_flow/loops.py:2072: note: Right operand is of type "DType? | None"
- jax/_src/lax/control_flow/loops.py:2073: error: Unused "type: ignore" comment  [unused-ignore]
- jax/_src/lax/control_flow/loops.py:2074: error: Unsupported left operand type for != (DType?)  [operator]
- jax/_src/lax/control_flow/loops.py:2074: note: Right operand is of type "DType? | None"
- jax/_src/lax/control_flow/loops.py:2075: error: Unused "type: ignore" comment  [unused-ignore]
- jax/_src/lax/control_flow/conditionals.py:241: error: Unsupported left operand type for != (PyTreeDef?)  [operator]
- jax/_src/numpy/linalg.py:1166: error: Redundant cast to "tuple[int, ...]"  [redundant-cast]
- jax/_src/numpy/lax_numpy.py:242: error: Unused "type: ignore" comment  [unused-ignore]
- jax/_src/numpy/lax_numpy.py:645: error: Unused "type: ignore" comment  [unused-ignore]
- jax/_src/numpy/lax_numpy.py:653: error: Unused "type: ignore" comment  [unused-ignore]
- jax/_src/numpy/lax_numpy.py:1273: error: Unused "type: ignore[call-overload]" comment  [unused-ignore]
- jax/_src/numpy/lax_numpy.py:1474: error: Redundant cast to "list[Any]"  [redundant-cast]
- jax/_src/numpy/lax_numpy.py:2138: error: Unused "type: ignore" comment  [unused-ignore]
- jax/_src/numpy/lax_numpy.py:3206: error: Unused "type: ignore" comment  [unused-ignore]
- jax/_src/numpy/lax_numpy.py:3305: error: Unused "type: ignore" comment  [unused-ignore]
- jax/_src/numpy/lax_numpy.py:3391: error: Unused "type: ignore" comment  [unused-ignore]
- jax/_src/numpy/lax_numpy.py:3598: error: Unused "type: ignore" comment  [unused-ignore]
- jax/_src/earray.py:60: error: Unused "type: ignore" comment  [unused-ignore]
+ Traceback (most recent call last):
+   File "", line 8, in <module>
+     sys.exit(console_entry())
+   File "/__main__.py", line 15, in console_entry
+     main()
+   File "/main.py", line 103, in main
+     res, messages, blockers = run_build(sources, options, fscache, t0, stdout, stderr)
+   File "/main.py", line 187, in run_build
+     res = build.build(sources, options, None, flush_errors, fscache, stdout, stderr)
+   File "/build.py", line 193, in build
+     result = _build(
+   File "/build.py", line 268, in _build
+     graph = dispatch(sources, manager, stdout)
+   File "/build.py", line 2950, in dispatch
+     process_graph(graph, manager)
+   File "/build.py", line 3348, in process_graph
+     process_stale_scc(graph, scc, manager)
+   File "/build.py", line 3449, in process_stale_scc
+     graph[id].type_check_first_pass()
+   File "/build.py", line 2314, in type_check_first_pass
+     self.type_checker().check_first_pass()
+   File "/checker.py", line 482, in check_first_pass
+     self.accept(d)
+   File "/checker.py", line 590, in accept
+     stmt.accept(self)
+   File "/nodes.py", line 927, in accept
+     return visitor.visit_decorator(self)
+   File "/checker.py", line 4993, in visit_decorator
+     self.visit_decorator_inner(e)
+   File "/checker.py", line 5020, in visit_decorator_inner
+     sig, t2 = self.expr_checker.check_call(
+   File "/checkexpr.py", line 1595, in check_call
+     result = self.check_call(
+   File "/checkexpr.py", line 1559, in check_call
+     return self.check_callable_call(
+   File "/checkexpr.py", line 1801, in check_callable_call
+     new_ret_type = self.apply_function_plugin(
+   File "/checkexpr.py", line 1256, in apply_function_plugin
+     return method_callback(
+   File "/plugins/functools.py", line 271, in partial_call_callback
+     result = ctx.api.expr_checker.check_call(
+   File "/checkexpr.py", line 1559, in check_call
+     return self.check_callable_call(
+   File "/checkexpr.py", line 1660, in check_callable_call
+     and callee.type_object().is_protocol
+   File "/types.py", line 1965, in type_object
+     assert isinstance(ret, Instance)
+ AssertionError: 
- jax/_src/scipy/special.py:2198: error: Argument 1 to "_expn3" has incompatible type "Array"; expected "int"  [arg-type]
- jax/_src/scipy/special.py:2199: error: Argument 1 to "_expn2" has incompatible type "Array"; expected "int"  [arg-type]
- jax/_src/scipy/special.py:2200: error: Argument 1 to "_expn1" has incompatible type "Array"; expected "int"  [arg-type]
- jax/_src/scipy/signal.py:760: error: Argument "type" to "detrend" has incompatible type "Literal[True] | str"; expected "str"  [arg-type]
- jax/_src/scipy/stats/norm.py:161: error: Redundant cast to "Array"  [redundant-cast]
- jax/experimental/host_callback.py:1092: error: Variable "jax.experimental.host_callback.XlaOp" is not valid as a type  [valid-type]
- jax/experimental/host_callback.py:1092: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#variables-vs-type-aliases
- jax/experimental/host_callback.py:1792: error: Variable "jax.experimental.host_callback.XlaLocalClient" is not valid as a type  [valid-type]

... (truncated 75 lines) ...

ibis (https://github.com/ibis-project/ibis)
+ ibis/backends/sql/datatypes.py: error: "int" not callable  [operator]
+ ibis/backends/sql/datatypes.py:31: error: Too many arguments for "__call__"  [call-arg]
+ ibis/backends/sql/datatypes.py:48: error: Too many arguments for "__call__"  [call-arg]
hauntsaninja commented 1 week ago 
from functools import partial
def foo(): ...
partial(partial, foo)()
github-actions[bot] commented 1 week ago

Diff from mypy_primer, showing the effect of this PR on open source code:

kopf (https://github.com/nolar/kopf)
- kopf/_core/actions/invocation.py:127: error: Incompatible types in assignment (expression has type "partial[partial[object | Coroutine[None, None, object]]]", variable has type "partial[object | Coroutine[None, None, object | None] | None]")  [assignment]
- kopf/_core/actions/invocation.py:127: error: Argument 1 to "run" of "Context" has incompatible type "partial[object | Coroutine[None, None, object | None] | None]"; expected "Callable[[VarArg(Any), KwArg(Any)], partial[object | Coroutine[None, None, object]]]"  [arg-type]
- kopf/_core/actions/invocation.py:127: note: "partial[object | Coroutine[None, None, object | None] | None].__call__" has type "Callable[[VarArg(Any), KwArg(Any)], object | Coroutine[None, None, object | None] | None]"

ibis (https://github.com/ibis-project/ibis)
+ ibis/backends/sql/datatypes.py: error: "int" not callable  [operator]
+ ibis/backends/sql/datatypes.py:31: error: Too many arguments for "__call__"  [call-arg]
+ ibis/backends/sql/datatypes.py:48: error: Too many arguments for "__call__"  [call-arg]
cdce8p commented 1 week ago

This PR seems to cause a false-positive with partial and not callable.

from functools import partial

class A:
    def __init__(self, var: str, b: int, c: int) -> None: ...

partial(A, str(1))  # false-positive
reveal_type(1)
test.py: error: "str" not callable  [operator]

Note: The error doesn't include a line number which made it difficult to default. Additionally, the __init__ signature seems to need three variables for the error to be emitted.

Furthermore, it isn't limited to str(...).

from functools import partial
class B: ...

class C:
    def __init__(self, var: B, b: int, c: int) -> None: ...

partial(C, B())  # false_positive
test.py: error: "B" not callable  [operator]
ilevkivskyi commented 1 week ago

Yes, this is a bug, fix in https://github.com/python/mypy/pull/17429