ramonhagenaars / jsons

🐍 A Python lib for (de)serializing Python objects to/from JSON
https://jsons.readthedocs.io
MIT License
288 stars 40 forks source link

Exception in Process Pool #92

Open wahello opened 4 years ago

wahello commented 4 years ago
When I use jsons in main process is ok, while json used in process pool raise Exception.
ramonhagenaars commented 4 years ago

Hi @wahello. I'm gonna need more information to reproduce your problem. Could you provide me the following information:

wahello commented 4 years ago

python version 3.7 jsons version 1.1.1

{'md5': 'aeb8399610e8a4fadee106aa590b294f', 'domain_id': '155', 'url': 'https://www.jianshu.com/nb/8834947', 'title': '随笔', 'domain': 'www.jianshu.com', 'category_id': '3', 'http_status': None, 'spider_status': '0', 'visits': '1', 'size': None, 'addtime': '2020-01-11 09:34:29', 'updatetime': '2020-01-11 09:34:29'}

Stack trace is

Exception in thread Thread-3:
Traceback (most recent call last):
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py", line 926, in _bootstrap_inner
    self.run()
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/pool.py", line 470, in _handle_results
    task = get()
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/connection.py", line 251, in recv
    return _ForkingPickler.loads(buf.getbuffer())
@dataclass
class DomainTaskModel():
    id: int
    domain_id: int
    domain: str
    title: Optional[str]
    url: str
    category_id: Optional[int]
    http_status: Optional[int]
    spider_status: Optional[int]
    visits: Optional[int]
    size: Optional[int]
    md5: str
    addtime: Optional[datetime]
    updatetime: Optional[datetime]