konradhalas / dacite

Simple creation of data classes from dictionaries.
MIT License
1.76k stars 106 forks source link

TypeError with Undeclared Type Annotations #15

Closed bpeake-illuscio closed 5 years ago

bpeake-illuscio commented 5 years ago

Sometimes it is necessary to declare types out-of-order. In this instance, quotes are put around the type to indicate the type has not yet been declared, but will be once the module is done importing.

Here is a trivial example:

>>> @dataclass
... class X:
...     text: str
...     y_data: "Y"
...     
>>> @dataclass
... class Y:
...     num: int
...     

Obviously, in this case, one could just reverse the declarations, but there are cases where types must remain in quotes, especially when handling cross-dependencies.

Having such a type declaration results in the following:

>>> x_dict = {
...     "text": "hello!",
...     "y_data": {
...         "num": 10
...     }
... }
>>> dacite.from_dict(x_dict)
Traceback (most recent call last):
  File "<input>", line 1, in <module>
TypeError: from_dict() missing 1 required positional argument: 'data'
>>> dacite.from_dict(X, x_dict)
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/Users/williampeake/venvs/spanreed-py-37/lib/python3.7/site-packages/dacite.py", line 93, in from_dict
    elif not _is_instance(field.type, value):
  File "/Users/williampeake/venvs/spanreed-py-37/lib/python3.7/site-packages/dacite.py", line 276, in _is_instance
    return isinstance(value, t)
TypeError: isinstance() arg 2 must be a type or tuple of types

Even explicitly having a Y class already in the dict throws the same error:

>>> x_dict = {
...     "text": "hello!",
...     "y_data": Y(10)
... }
>>> dacite.from_dict(X, x_dict)
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/Users/williampeake/venvs/spanreed-py-37/lib/python3.7/site-packages/dacite.py", line 93, in from_dict
    elif not _is_instance(field.type, value):
  File "/Users/williampeake/venvs/spanreed-py-37/lib/python3.7/site-packages/dacite.py", line 276, in _is_instance
    return isinstance(value, t)
TypeError: isinstance() arg 2 must be a type or tuple of types

Thanks for your time. This is an awesome library, and I plan to make fairly heavy use of it for serializing / deserializing dataclasses from json.

konradhalas commented 5 years ago

@bpeake-illuscio this is very interesting case.

It looks that there is no easy way to transform a forward reference to a real type. We need globals/locals from context of given data class - this is how typing.get_type_hints() works. You can pass globals/locals to this function and it will transform "strings" to types for you. We can add globals/locals to Config but it's not the most beautiful API. Instead of this we can just pass a dictionary with string to type mapping (although this is the same solution, globals/locals are just dicts too).

I add one more simple case for this issue, it's also broken:

@dataclass
class Node:
    value: int
    parent: Optional['Node']

I have to think about it.

sollyucko commented 5 years ago

Why not just use typing.get_type_hints()? Also, another problem with the current behavior is with from __future__ import annotations. I haven't tried it yet, but it should trigger this error for all annotations.

bpeake-illuscio commented 5 years ago

Nested declarations do not work super seamlessly with get_type_hints()

>>> from typing import get_type_hints
>>> from dataclasses import dataclass
>>> 
>>> def nested():
...     class X:
...         y: "Y"
...     
...     class Y:
...         text: str
...     
...     get_type_hints(X)
...     
>>> nested()
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "<input>", line 8, in nested
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/typing.py", line 973, in get_type_hints
    value = _eval_type(value, base_globals, localns)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/typing.py", line 260, in _eval_type
    return t._evaluate(globalns, localns)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/typing.py", line 464, in _evaluate
    eval(self.__forward_code__, globalns, localns),
  File "<string>", line 1, in <module>
NameError: name 'Y' is not defined
sollyucko commented 5 years ago

We can pass locals and globals. Also, do we have any better way to do it?

konradhalas commented 5 years ago

Fixed in https://github.com/konradhalas/dacite/pull/27