Include a sequence and combine alternative which is typed

[robjhornby]

I am interested in a typed version of parsy and would find it useful, the effort in typed-parsy so far is a nice improvement. I read this blog post on this topic and had an idea for an alternative to seq and combine for the *args synatax (not **kwargs unfortunately). I have something working with Pyright (I haven't tried mypy) - see the linked PR below

By using tuple type unpacking from PEP 646, it's possible to add type information to both of these functions which enable creating a sequence of parsers with a flat tuple result type:

• A function which joins the results of two parsers into a tuple: Parser[A], Parser[B] -> Parser[Tuple[A, B]]. I've called this join for now. (This one doesn't need tuple type unpacking - it is just the Parser.__and__ function.)
• A function which takes a tuple result and appends a second parser's result to the end: Parser[Tuple[A, B]], Parser[C] -> Parser[Tuple[A, B, C]]. I've called this append for now.

And it's also possible to create a combine method which expects function whose arguments have the same type as the parser's result tuple. The type signature is like this:

from typing import TypeVar

from typing_extensions import TypeVarTuple, Unpack

OUT2 = TypeVar("OUT2")
OUT_T = TypeVarTuple("OUT_T")

# ... in the Parser class
    def combine(self: Parser[Tuple[Unpack[OUT_T]]], combine_fn: Callable[[Unpack[OUT_T]], OUT2]) -> Parser[OUT2]:
        ...

The benefit is that Pyright will compare the Parser result type with the combine_fn arguments and show an issue when they don't match. It also doesn't need the extra step of indexing tuples in a lambda function as shown in the blog post.

I've made these changes in a fork and they seem to be working nicely with Pyright - it's showing me mismatches between a sequence of parsers' results and a function's arguments with useful messages.

For example, in the screenshot below, the demo function expects its first argument to have a type of str. A parser of type Parser[int, str, bool] can be made with the join and append functions. Then where combine is called, VSCode shows that there's a problem - the parser result type doesn't match demo's parameter type in the first argument.

[robjhornby]

Other comments:

• I'm not sure having two functions join and append is the best interface, particularly if the sequence is short, but their different type signatures make it tricky to do something else.
• I looked for an equivalent for unpacking kwargs, hoping that seq with keywords and combine_dict could be typed, but found nothing. PEP 646 mentions that they don't know a use case for unpacking kwargs, maybe parser libraries could be a good use case.
• Pyright and mypy have some support for typing functools.partial which may be another route for supporting **kwargs sequences. A function being called via partial with one kwarg at a time, then finally being called as a complete function is very similar to creating a sequence of parsers and assigning their results to a keyword one at a time, then finally passing all those kwargs to a function.

[spookylukey]

Hi @robjhornby thanks for your input.

I currently have no plans to further develop this project, for the reasons I described in that blog post, and I'm afraid I'm not likely to have the bandwidth in the foreseeable future to look at your PR in anything like a timely fashion.

However, if you would like to continue work on it, I'm very happy for you to do so. I could make you an admin on it if you would like? My only request is that you make clear the status of the project, and the fact that it has a different maintainer

[robjhornby]

Hi @spookylukey, thanks for the quick response

I'm not sure I would have a clear goal for this project, so I'm happy to keep experimenting in a fork for now, and if I make some progress I will come back to this repo - thanks for the offer/suggestion.

If I make some progress, are you open to including more type annotations in parsy, even if you're not pursuing this fully typed version? The idea of annotations as "clear and concise documentation for humans that pop up in a code editor" is also what I'm aiming for, as opposed to a complete statically typed solution.

[spookylukey]

If I make some progress, are you open to including more type annotations in parsy, even if you're not pursuing this fully typed version? The idea of annotations as "clear and concise documentation for humans that pop up in a code editor" is also what I'm aiming for, as opposed to a complete statically typed solution.

Yes, I'm very interested in type hints along those lines. I already added quite a lot, covering all the main methods - https://github.com/python-parsy/parsy/pull/69 released in v 2.1 - but if there are things I missed or ways it can be improved please go ahead.