Open wahello opened 4 years ago
Hi @wahello. I'm gonna need more information to reproduce your problem. Could you provide me the following information:
python version 3.7 jsons version 1.1.1
{'md5': 'aeb8399610e8a4fadee106aa590b294f', 'domain_id': '155', 'url': 'https://www.jianshu.com/nb/8834947', 'title': '随笔', 'domain': 'www.jianshu.com', 'category_id': '3', 'http_status': None, 'spider_status': '0', 'visits': '1', 'size': None, 'addtime': '2020-01-11 09:34:29', 'updatetime': '2020-01-11 09:34:29'}
Stack trace is
Exception in thread Thread-3:
Traceback (most recent call last):
File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/pool.py", line 470, in _handle_results
task = get()
File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/connection.py", line 251, in recv
return _ForkingPickler.loads(buf.getbuffer())
@dataclass
class DomainTaskModel():
id: int
domain_id: int
domain: str
title: Optional[str]
url: str
category_id: Optional[int]
http_status: Optional[int]
spider_status: Optional[int]
visits: Optional[int]
size: Optional[int]
md5: str
addtime: Optional[datetime]
updatetime: Optional[datetime]
When I use jsons in main process is ok, while json used in process pool raise Exception.