konradhalas / dacite

Simple creation of data classes from dictionaries.
MIT License
1.72k stars 107 forks source link

TypeError when dataclass contains list of other dataclasses #16

Closed bpeake-illuscio closed 5 years ago

bpeake-illuscio commented 5 years ago

It looks like dacite errors out when given a dict with a list of loaded dataclasses already in it. Because of the way json.loads works, this is something that pops up when trying to load dataclasses directly. json.loads starts at the deepest object, and works its way back up. Dataclasses with a list of other dataclasses will have each object in the list loaded first.

Here is an example that manifests the bug:

>>> import dacite
>>> 
>>> from dataclasses import dataclass, field
>>> from typing import List
>>> 
>>> @dataclass
... class X:
...     text: str = "default"
...     
>>> @dataclass
... class Y:
...     x: X = X()
...     
>>> @dataclass
... class Y:
...     x_list: List[X] = field(default_factory=list)
... 
>>> y_dict = {"x_list": [X(), X()]}
>>> dacite.from_dict(Y, y_dict)
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/Users/williampeake/venvs/spanreed-py-37/lib/python3.7/site-packages/dacite.py", line 78, in from_dict
    field=field,
  File "/Users/williampeake/venvs/spanreed-py-37/lib/python3.7/site-packages/dacite.py", line 189, in _inner_from_dict_for_collection
    ) for item in data)
  File "/Users/williampeake/venvs/spanreed-py-37/lib/python3.7/site-packages/dacite.py", line 189, in <genexpr>
    ) for item in data)
  File "/Users/williampeake/venvs/spanreed-py-37/lib/python3.7/site-packages/dacite.py", line 61, in from_dict
    _validate_config(data_class, data, config)
  File "/Users/williampeake/venvs/spanreed-py-37/lib/python3.7/site-packages/dacite.py", line 101, in _validate_config
    _validate_config_data_key(data, config, 'remap')
  File "/Users/williampeake/venvs/spanreed-py-37/lib/python3.7/site-packages/dacite.py", line 122, in _validate_config_data_key
    input_data_keys = set(data.keys())
AttributeError: 'X' object has no attribute 'keys'

And here is an example of when this pops up in practical code:

>>> import json
>>> from typing import Type
>>> 
>>> def dataclass_hook(obj):
...     class_index = {dc.__name__: dc for dc in [X, Y]}
...     try:
...         type_name = obj['_type']
...     except (KeyError, IndexError):
...         return obj
...     try:
...         data_type = class_index[type_name]
...     except KeyError:
...         return obj
...     # lets print each step so we can see what order objects are being loaded,
...     # and what values look like when the error is thrown
...     print(data_type, obj)
...     return dacite.from_dict(data_type, obj)
... 
>>> data = {
...     "_type": "Y",
...     "x_list": [
...         {"_type": "X", "text": "value one"},
...         {"_type": "X", "text": "value two"},
...     ]
... }
>>> json_string = json.dumps(data)
>>> loaded = json.loads(json_string, object_hook=dataclass_hook)
<class '__main__.X'> {'_type': 'X', 'text': 'value one'}
<class '__main__.X'> {'_type': 'X', 'text': 'value two'}
<class '__main__.Y'> {'_type': 'Y', 'x_list': [X(text='value one'), X(text='value two')]}
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/__init__.py", line 361, in loads
    return cls(**kw).decode(s)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/decoder.py", line 353, in raw_decode
    obj, end = self.scan_once(s, idx)
  File "<input>", line 14, in dataclass_hook
  File "/Users/williampeake/venvs/spanreed-py-37/lib/python3.7/site-packages/dacite.py", line 78, in from_dict
    field=field,
  File "/Users/williampeake/venvs/spanreed-py-37/lib/python3.7/site-packages/dacite.py", line 189, in _inner_from_dict_for_collection
    ) for item in data)
  File "/Users/williampeake/venvs/spanreed-py-37/lib/python3.7/site-packages/dacite.py", line 189, in <genexpr>
    ) for item in data)
  File "/Users/williampeake/venvs/spanreed-py-37/lib/python3.7/site-packages/dacite.py", line 61, in from_dict
    _validate_config(data_class, data, config)
  File "/Users/williampeake/venvs/spanreed-py-37/lib/python3.7/site-packages/dacite.py", line 101, in _validate_config
    _validate_config_data_key(data, config, 'remap')
  File "/Users/williampeake/venvs/spanreed-py-37/lib/python3.7/site-packages/dacite.py", line 122, in _validate_config_data_key
    input_data_keys = set(data.keys())
AttributeError: 'X' object has no attribute 'keys'