Investigate Type Annotation

[iwoloschin]

Is your feature request related to a problem? Please describe. I just started exploring pyinfra and so far I like it a lot. I happen to use VSCode with the Pylance plugin. This appears to throw a ton of red squiggles on anything decorated with the @operation decorator if Type Checking Mode is set to basic. The completion/help for any function arguments is also broken. I suspect the issue is related to https://github.com/microsoft/pyright/issues/774, but I'm not sure how to deal with how the @operation decorator adds global arguments.

Describe the solution you'd like I suspect providing type annotations for the entire project is out of scope (for now... 😄 ), but it would be interesting to explore how to better support some basic type annotations. This may be more of a v2 problem though.

[Fizzadar]

Hi @iwoloschin! I'm super keen on getting type annotations throughout pyinfra, but as you say this is essentially on hold until v2 which will drop python 3 support.

Also interesting is that pyright issue - the operation decorator (currently) uses *args, **kwargs which would still have the same issue. The solution there certainly looks like an option, should be able to carry the type annotations of the wrapped function into the wrapped one!

[iwoloschin]

I played around with that pyright workaround, it does work for passing the operation's normal arguments forward through the @operation decorator. This can be pretty easily added in a .pyi file.

Unfortunately, that doesn't do anything to help with the global arguments. The only way I've found to get that to work is to manually define them (again, I did this in a separate .pyi file). This works, but it is fairly tedious and probably prone to errors should the set of global arguments ever change. I think the proper issue to follow for something like this would be https://github.com/python/typing/issues/193, but it is too early and I've only had one cup of coffee so I might be misreading that!

Separately, it seems that host, inventory, and state are not real modules, which also trips up pylance (and presumably mypy?). I'm not sure what, if any, the right fix is there for now, but mentioning it here as it is related to typing.

Anyways, I'll probably continue to play with typing stubs (https://mypy.readthedocs.io/en/stable/stubs.html), would you be interested in PRs?

[Fizzadar]

It does indeed look like https://github.com/python/typing/issues/193 might cover that use case!

Long term I'd like type annotations inline but I'm open to PRs implementing stub as they can be merged in easily once py2 is dropped :)

Regarding the host/inventory/state - these are implemented as pseudo modules with a base class, might be possible to leverage that somehow?

[iwoloschin]

I've made a PR #444 to try getting the bare minimum typing information for supporting the @operation decorator. I haven't done anything with typing stubs before so it is possible I'm missing something, but it appears to work for me.

If you're comfortable merging in #444 then I can start adding typing stubs for all of the operations. I'll look into the host/inventory/state modules too but I think getting the operations typed would be the biggest improvement so I'll probably focus there first.

[mrahtz]

@Fizzadar I noticed you referenced python/typing#193 on variadic generics in this thread. Heads up that we've been working on a draft of a PEP for this in PEP 646. If this is something you still care about, take a read and let us know any feedback in this thread in typing-sig. Thanks!

[karlicoss]

I've found lack of types not that bad since there are decent docs and pyinfra seems well tested. But for my own custom deploys I'm more worried about messing them up. For anyone who wants to at least type their own deploys, you can use this trick:

from typing import TypeVar, Callable
F = TypeVar('F', bound=Callable)
if typing.TYPE_CHECKING:
    # this will expose the wrapped function's type to mypy
    def deploy(f: F) -> F:
        pass 
else:
    from pyinfra.api import deploy

Then stuff like this will be type checked:

@deploy
def my_operation(*, arg1: str, arg2: int, **kwargs): 
    ...

You can also probably use it for typing some of pyinfra's API on ad-hoc basis

[Fizzadar]

Typing is now in! Not 100% but mypy is part of CI and types can gradually be introduced. The global operation args are typed.