Cannot use typing.Generic with pytypes

[udim]

Using pytypes 1.0b5:

Python 3.7.3rc1 (default, Mar 13 2019, 11:01:15) 
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import typing
>>> import pytypes
>>> class PCollection(typing.Generic[typing.TypeVar('T')]):
...   pass
... 
>>> PCollection()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/google/home/ehudm/virtualenvs/beam-py37/lib/python3.7/site-packages/pytypes/__init__.py", line 567, in __Generic__new__
    if cls.__origin__ is None:
AttributeError: type object 'PCollection' has no attribute '__origin__'

[udim]

This is the version of typing I have installed: https://github.com/python/cpython/blob/v3.7.3rc1/Lib/typing.py

[Stewori]

This is a manifestation of #40. That issue is significant work, so don't expect a fix too soon. However, there is steady progress and current master is readily miles better than 1.0b5. Especially various __origin__-related issues were fixed, so its worth a try for your usecase.

[Stewori]

It would be good if you could test this using current pytypes master. If it fails with master I would reopen in order to keep track and assert that my work on pytypes' Python 3.7 compatibility actually solves this issue as well.

[udim]

It works on master.

[sobolevn]

Sadly, it does not work for me.

Python 3.7.4 (default, Aug 10 2019, 01:56:59) 
[Clang 10.0.1 (clang-1001.0.46.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from pytypes import typechecked
>>> from typing import Generic, TypeVar
>>> T = TypeVar('T')
>>> class My(Generic[T]):
...     def __init__(self, arg: T) -> None:
...         self._arg = arg
... 
>>> @typechecked
... def test(m: My[int]) -> None:
...     print(m)
... 
>>> test(My(1))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
pytypes.exceptions.InputTypeError: 
  __main__.test
  called with incompatible types:
Expected: Tuple[My[int]]
Received: Tuple[My]

I have installed pytypes from master on this commit: https://github.com/Stewori/pytypes/blob/b7271ec654d3553894febc6e0d8ad1b0e1ac570a/setup.py Version shown: pytypes==1.0b5.post38

[Stewori]

Your example is beyond the scope of typechecking, it's about type inference. Typechecking in sense of @typechecked is performed w.r.t. given types. If you use test(My[int](1)) it should work. That is the intended use by PEP 484 AFAIK. Pytypes can actually do some type inference, e.g. get types from default args or write designated runtime observations to stubfiles. However it does not perform the on-the-fly type inference your example would require. Does this work with other typecheckers?

I think it might be doable to perform this type of type inference but it would be a significantly new feature. If you like, file it as a feature request in a separate issue. Bugfixing and Python 3.7+ support are currently dominant priorities, so don't expect feature requests to be handled soon (unless you make a PR). Still, it might be worth to track it...

[sobolevn]

Yes, it works with mypy. It would be awesome to have something similar, but for runtime.

[Stewori]

It wouldn't be feasible without any decorator or agent on My.__init__ (or inderectly e.g. from a decorator over My) that hooks in to infer the type of T when the call happens. There are more questions to be answered: Should types be inferred on every call or only on __init__? The decorator could control this. Please open a separate issue for this.

[Stewori]

In python/mypy#949 you wrote:

I mean: are there any reliable ways to ensure that x is instance of List[int] or MyCustomGeneric[str]?

Note that that demand is rather different from what you bring up here. You can use pytypes.is_of_type to check if x is instance of List[int] or MyCustomGeneric[str]. is_of_type nowadays has more optional args than are documented there (doc needs an update). E.g. you can provide a dictionary as optional argument bound_typevars to that function, e.g. {T: int} to tell it parameters for free typevars.

[sobolevn]

Oh, great @Stewori! That's exactly what I need. But, sadly it does not work either.

>>> X = TypeVar('X')
>>> class My(Generic[X]):
...     def __init__(self, arg: X) -> None:
...         self._arg = arg
... 
>>> is_of_type(My(1), My[str])
False
>>> is_of_type(My(1), My[int])
False

Any ideas?

[Stewori]

is_of_type(My[int](1), My[int]) is_of_type(My[X](1), My[int], bound_typevars={X: int}) both work as expected. Without type inference, types are handled in strictly declarative fashion, i.e. My and My[int] are really distinct types. That's actually not bad as it requires to specify the type more explicitly. Explicit is better than implicit, right? I admit it is a little bit disappointing that is_of_type(My(1), My[int], bound_typevars={X: int}) does not work. If I find time I may revisit that one.

[Stewori]

Also consider My[type(1)](1) or My[pytypes.deep_type(1)](1) to assess the type in more dynamic fashion.