Add type hints

[devRMA]

Use a static type checker called mypy. In libs, I think it's very important to have type hints, so that when users are going to use the lib, code linters can help, knowing exactly what kind of parameters to receive, and what the return type is.

The ideal solution would be to add type hints as specified by PEP 484, PEP 526, PEP 544, PEP 586, PEP 589, and PEP 591 directly on your code.

[devRMA]

I even noticed that you had already added mypy to the requirements. Just need to do static typing.

https://github.com/JulienPalard/Pipe/blob/dd179c8ff0aa28ee0524f3247e5cb1c51347cba6/requirements.txt#L3-L4

[JulienPalard]

Probably blocked by https://github.com/python/mypy/issues/11833, as I'll need something like:

P = ParamSpec("P")
I = TypeVar("I")
O = TypeVar("O")

class Pipe(Generic[P, I, O]):
    def __init__(self, function: Callable[Concatenate[Iterable[I], P], Iterator[O]]) -> None:
        ...

as later the __call__ arguments are P. (The Iterable[I] being passed to the function via __ror__.)

But yes I agree it would be very nice to be able to write something like:

T = TypeVar("T")
take: Pipe[T, int, T]

to express than take takes an int, and yields the same type that the feeded type.

[JulienPalard]

Now blocked by mypy issue 1484 as we need a way to cleanly decribe functools.partial, to properly describe:

def __call__(self, *args, **kwargs):
    return Pipe(
        lambda iterable, *args2, **kwargs2: self.function(
            iterable, *args, *args2, **kwargs, **kwargs2
        )
    )

[devRMA]

It's more complex than I thought 😳

[0xMochan]

Now blocked by mypy issue 1484 as we need a way to cleanly decribe functools.partial, to properly describe:

def __call__(self, *args, **kwargs):
    return Pipe(
        lambda iterable, *args2, **kwargs2: self.function(
            iterable, *args, *args2, **kwargs, **kwargs2
        )
    )

I'm pretty sure this is nearly impossible without being able to represent HKTs (Higher-Kinded Types) in the Python type system.

[fabiob]

@JulienPalard and @devRMA, would you share the code you write so far in a branch or fork? I would love to take a swing at it and try to help you.

[JulienPalard]

@fabiob pushed my typing branch, see the pyi file. I know it's not much and does not works.

What we really need is a clean way to type functools.partial, as it's what's pipe does a lot.

[p-98]

Seems like https://github.com/python/mypy/issues/1484 was completed with https://github.com/python/mypy/pull/16939.

Does the implementation provide what you need?

[JulienPalard]

I don't think so, they implemented a mypy plugin to handle the single functools.partial function, I really mean there's an actual if fullname == "functools.partial": in the implementation.

Looks like there's still no way to type annotate functools.partial equivalents.

[JulienPalard]

Currently this works:

from collections.abc import Callable
from typing import Concatenate, Generic, ParamSpec, TypeVar

P = ParamSpec('P')
T = TypeVar('T')
U = TypeVar('U')

class Pipe(Generic[U, P, T]):
    def __init__(self, function: Callable[Concatenate[U, P], T]):
        self.function = function

    def run(self, arg1: U) -> T:
        return self.function(arg1)

def simple_annotated_function(value: int) -> str:
    assert isinstance(value, int)
    return "hello"

def main() -> None:
    p = Pipe(simple_annotated_function)
    reveal_type(p)  # demo.Pipe[builtins.int, [], builtins.str]
    reveal_type(p.function) # def (builtins.int) -> builtins.str
    reveal_type(p.run) # def (arg1: builtins.int) -> builtins.str
    p.run("coucou") # Argument 1 to "run" of "Pipe" has incompatible type "str"; expected "int"

main()

But it feels like it's only 1/10 of the work, as we need some way to "substract" parameters from a paramspec.

[JulienPalard]

Starting a new discussion here: https://github.com/python/mypy/issues/17481 to track this.