Open mitar opened 6 years ago
I just saw this issue browsing the repo, but are you sure that Dict[str, List]
is a valid type? [EDIT: turns out it is, see my next comment] I'm pretty sure that typing.List
is meant to be used with a type parameter, like List[int]
. If you don't want to specify the types inside, you can do List[Any]
or list
(i.e. the Python built-in).
List
should be equivalent to List[Any]
. In any case, this should not throw and error.
My bad, it looks like you were right! PEP 484 says:
If a generic type is used without specifying type parameters, they (sic) assumed to be Any:
def use_map(m: Mapping) -> None: # Same as Mapping[Any, Any] ...
So this usage should be supported.
I think this originates from deep_type
lacking support for defaultdict
. We'll have to improve deep_type
to handle classes from collections properly.
Re missing parameters also see pytypes.check_unbound_types
and pytypes.strict_unknown_check
.
Finally let me remind you to post stacktraces with having called pytypes.enable_clean_traceback(False)
. For completeness and convenience, here the full stacktrace:
Traceback (most recent call last):
File "/data/workspace/linux/pytypes/quick_test/issue_52.py", line 12, in <module>
print pytypes.is_of_type(collections.defaultdict(list), typing.Dict[str, typing.List])
File "/data/workspace/linux/pytypes/pytypes/type_util.py", line 1909, in _isinstance
bound_typevars_readonly, follow_fwd_refs, _recursion_check)
File "/data/workspace/linux/pytypes/pytypes/type_util.py", line 1774, in _issubclass
bound_typevars_readonly, follow_fwd_refs, _recursion_check)
File "/data/workspace/linux/pytypes/pytypes/type_util.py", line 1801, in _issubclass_2
_recursion_check)
File "/data/workspace/linux/pytypes/pytypes/type_util.py", line 1145, in _issubclass_Mapping_covariant
return issubclass(subclass, superclass)
File "/usr/local/lib/python2.7/dist-packages/typing.py", line 1243, in __subclasscheck__
raise TypeError("Parameterized generics cannot be used with class "
TypeError: Parameterized generics cannot be used with class or instance checks
Finally let me remind you to post stacktraces with having called
pytypes.enable_clean_traceback(False)
.
Ah, sorry. If forget about this.
Ah, sorry. If forget about this.
Actually it's my fault that this is still not pointed out in the readme. Will add that hint...
Currently the code in deep_type
that resolves dict
is:
elif res == dict:
if len(obj) == 0:
return Empty[Dict]
if max_sample == -1 or max_sample >= len(obj)-1 or len(obj) <= 2:
try:
# We prefer a view (avoid copy)
tpl1 = tuple(_deep_type(t, checked, checked_len2, depth-1, None, get_type) \
for t in obj.viewkeys())
tpl2 = tuple(_deep_type(t, checked, checked_len2, depth-1, None, get_type) \
for t in obj.viewvalues())
except AttributeError:
# Python 3 gives views like this:
tpl1 = tuple(_deep_type(t, checked, checked_len2, depth-1, None, get_type) for t in obj.keys())
tpl2 = tuple(_deep_type(t, checked, checked_len2, depth-1, None, get_type) for t in obj.values())
else:
try:
kitr = iter(obj.viewkeys())
vitr = iter(obj.viewvalues())
except AttributeError:
kitr = iter(obj.keys())
vitr = iter(obj.values())
ksmpl = []
vsmpl = []
block = (len(obj) // max_sample)-1
# I know this method has some bias towards beginning of iteration
# sequence, but it's still more random than just taking the
# initial sample and better than O(n) random.sample.
while len(ksmpl) < max_sample:
if block > 0:
j = random.randint(0, block)
k = random.randint(0, block)
while j > 0:
next(vitr) # discard
j -= 1
while k > 0:
next(kitr) # discard
k -= 1
ksmpl.append(next(kitr))
vsmpl.append(next(vitr))
tpl1 = tuple(_deep_type(t, checked, checked_len2, depth-1, None, get_type) for t in ksmpl)
tpl2 = tuple(_deep_type(t, checked, checked_len2, depth-1, None, get_type) for t in vsmpl)
res = Dict[Union[tpl1], Union[tpl2]]
where res
holds the result of type(obj)
. We need to refine it to consider defaultdict and ideally also the other dict-like types from collections, see https://docs.python.org/2/library/collections.html.
I think this is mostly a question of improving the entrance elif res == dict:
. Would elif issubclass(res, dict):
do it? I suspect this might target some unwanted types as well but I don't know right now. Ideas?
Specifically for defaultdict
I suppose we'd want to include the default type into the resulting valuetypes union. Can someone contribute a code snippet to achieve this?
It seems also collections.OrderedDict
fails with currently released code in the same way.
Sadly I do not have answers to questions you raised above. I guess issubclass(res, dict)
would be OK to do.
Yes, all classes from collections are currently not supported. deep_type
does not work with subclasses of builtin types. E.g. if someone defines (can tuple be subclassed? If not, this still illustrates the issue for other types that can)
class mytuple(tuple):
blah
deep_type(mytuple(1, 2, 3)) # would result in `mytuple`
would we want this to result in Tuple[int, int, int]
? I.e. let deep_type
use the closest PEP 484 type to declare a custom type? But that would loose the info that it is not an ordinary tuple but actually a mytuple
.
Let's say the author of mytuple
also created an appropriate generic type MyTuple[...]
. I would like to have a way she can register MyTuple
in pytypes such that it can utilize it to represent types of mytuple
objects. We could then have PEP 484 generics like OrderedDict
which can simply subclass Dict
. DefaultDict
already exists: https://www.python.org/dev/peps/pep-0484/#instantiating-generic-classes-and-type-erasure. Is there actually an official PEP 484 way to describe OrderedDict
? I think it's just Dict
. What if a function really requires an ordered dict rather than a dict?
Okay, I suggest the following:
Lets add support for datattypes in collections explicitly in the usual fashion.
Sooner or later I would like to modulize deep_type
such that we have a set of objects of kind of class typer
that contain the logic to find the PEP 484 type for an object of a given ordinary type. We would then have typers for dict, defaultdict, tuple etc and a user can simply inject support for mytuple
by adding a corresponding typer to that list. However, that's more the longterm plan.
typer
could also serve to place optionally some custom logic for type checking or subtype checking specific types. This would also somewhat modulize the complex logic in is_subtype
.
I found out that this occurs even with a regular dict: #56
I would hope to get
True
instead.