Better support for decorators

[nickroeker]

Pyre claims to try to match compatibility with mypy, which has a documented usage of decorators that Pyre can't seem to handle (tested in version 0.0.57).

See here: https://mypy.readthedocs.io/en/stable/cheat_sheet_py3.html

This code is expected to type-check correctly,

F = TypeVar('F', bound=Callable[..., Any])
def decorator_args(url: str) -> Callable[[F], F]: ...

But Pyre has the following to state about this line,

Invalid type variable [34]: The type variable `Variable[F]` isn't present in the function's parameters.

This is true for more complicated and/or concrete cases as well.

[nickroeker]

I'm not expecting this to be an easy or simple fix. If more examples of decorator typing are critically needed, I can work on providing a few (though my use-cases aren't very much more complicated than the above).

Please note that decorators without args work just fine, which could be type-hinted like so (also from the above mypy link):

def bare_decorator(func: F) -> F: ...

From my perspective, this works while the other doesn't because Pyre is requiring the TypeVar to be in the parameter list. For a decorator with args, the TypeVar can't be in the parameter list of the outermost function.

[grievejia]

I believe the behavior is intentional. Relevant discussions can be found in this mailing thread.

The core issue here is that there is no facilities in standard Python to specify the scope of type variables, which lead to ambiguities on the bounds of generic types: does your decorator_args function have the type forall F. str -> (F -> F), or does it have the type str -> (forall F. F -> F)? Note that those two types have very different meanings. What mypy does is to interpret it as the latter, i.e. only quantify the return type. But there are also cases where it heuristically chooses to quantify both the parameter and the return types. In contrast, pyre prioritize consistency and choose to always quantify params&returns.

If you want your decorator to be accepted by both mypy and pyre, you can write the return type of decorator_args as a callback protocol, as suggested in the email thread:

F = TypeVar('F', bound=Callable[..., Any])
class MyDecorator(Protocol):
  def __call__(self, __f: F) -> F: ...
def decorator_args(url: str) -> MyDecorator: ...

This way, there won't be any ambiguities on where F is bound.

Also maybe relevant: note that for decorators, pyre additionally supports PEP 612 which allows you to assign more precise typing to __f without resorting to Any.