Suggest using Protocol when user tries to use keyword args with Callable

[ArgentFalcon]

If I define a passed in function using the Callable syntax, mypy complains if I then call a keyword argument on that function. This is makes writing things like decorators problematic

Sample code:

from typing import Callable

def test_func(a, b, c=0, d=0):
    # type: (int, int, int, int) -> int
    return a + b + c + d

def check_func(method):
    # type: (Callable[[int, int, int], int]) -> int
    return method(1, 2, d=5)

print(check_func(test_func))

$ mypy sample_python.py
sample_python.py: note: In function "check_func":
sample_python.py:9: error: Unexpected keyword argument "d"

As an aside here, I the following function code also raises mypy errors:

from typing import Callable

def test_func(a, b, *, c, d):
    # type: (int, int, int, int) -> int
    return a + b + c + d

def check_func(method):
    # type: (Callable[[int, int, int, int], int]) -> int
    return method(1, 2, c=5, d=10)

print(check_func(test_func))

[gvanrossum]

You can use Callable[..., int] here, with literally three dots.

--Guido (mobile)

[ArgentFalcon]

Is there any way to be more specific?

[gvanrossum]

Sorry, there's nothing more specific; it's hard to fit this in Python's syntactic constraints on the X[Y] notation. For decorators that don't change the signature of the thing they're decorating, you can use this:

F = TypeVar('F', bound=Callable[..., Any])
def decorator(func: F) -> F:
    def wrapper(*args, **kwds):
        return func(*args, **kwds)
    return cast(F, wrapper)

But obviously this doesn't work if the decorator changes the signature, nor if the wrapper needs to pass a keyword arg.

[willtalmadge]

I hit this error when I was trying to constrain the types of injected dependencies, so my solution may not apply to your decorator problem. But, this thread comes up first on google for this error so I'm just adding this as a potential solution others might find helpful. I just replaced my anonymous type with an interface by doing the moral equivalent to the following transformation of your code example.

from abc import ABC, abstractmethod

class FourNumGizmo(ABC):
    @abstractmethod
    def __call__(
            self,
            a: int,
            b: int,
            c: int = 0,
            d: int = 0
    ) -> int:
        pass

class MyAdder(FourNumGizmo):
    def __call__(
            self,
            a: int,
            b: int,
            c: int = 0,
            d: int = 0
    ) -> int:
        return a + b + c + d

def check_four_num_gizmos(gizmo: FourNumGizmo) -> int:
    return gizmo(1, 2, d=5)

print(check_four_num_gizmos(MyAdder()))

[pkch]

Might be worth changing the error message to something that explains that Callback does not support keyword arguments, and a link to this bug for alternatives.

[ethanhs]

I think we should suggest using Protocols like we do here https://mypy.readthedocs.io/en/latest/protocols.html#callback-protocols. Perhaps that section could be made more generic? Then we can link to that if someone tries to use keywords on a Callable.

[mahmoudajawad]

Coming across this issue, and following GvR explanation on how Callable can't be used with keyword-arguments, I think @pkch, @ethanhs suggestions are spot-on, and mypy can return an error message when a function typed as Callable is used with keyword-arguments, but still suggest the better alternative of using Protocol which I just tried and it did the work perfectly.

P.s.: The docs are referring to importing Protocol from typing_extensions where I did from typing. Is this something worth looking at and updating in the docs, or that importing Protocol is supposed to be from typing_extensions?

[gvanrossum]

You can always import Protocol from typing_extensions, whereas you need at least Python 3.8 to be able to import it from typing. That's the way it goes.

[LefterisJP]

With Python 3.8 and the Protocol option as defined in the docs: https://mypy.readthedocs.io/en/stable/protocols.html#callback-protocols this issue should now be closed, no? I came here having the same question as the OP and found the answer of using typing.Protocol more than sufficient and easy to use in our project.

[JelleZijlstra]

The OP's example now produces this output:

main.py:9: error: Unexpected keyword argument "d"

As some people suggested above, we should add a note pointing towards using Protocol. Going to retitle the issue to reflect that suggestion.