search

scrapy / scrapyd

A service daemon to run Scrapy spiders
https://scrapyd.readthedocs.io/en/stable/
BSD 3-Clause "New" or "Revised" License
2.94k stars 571 forks source link

scrapy run all right,but when i run spider using scrapyd,there is UnicodeEncodeError #332

Closed Rkndros closed 5 years ago

Rkndros commented 5 years ago

UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-7: ordinal not in range(128) like that. But using scrapy is ok.

Digenis commented 5 years ago

Hi @thereds2007, please post the full traceback

Rkndros commented 5 years ago

ERROR: Error processing {'link': u'https://feng.ifeng.com/c/7mbXjIVbUN1', 'time': u'\t\u51e4\u51f0\u8d44\u8baf\r\n 2019-05-11 22:30:30', 'title': u'\u8d38\u6613\u6218\u6ca1\u6709\u8d62\u5bb6'} Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 655, in _runCallbacks current.result = callback(current.result, *args, **kw) File "build/bdist.linux-x86_64/egg/SocialNet/pipelines.py", line 11, in process_item UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-6: ordinal not in range(128)

i think it is the problem of encoding.But ,what is strange is that scrapy works fine.

Rkndros commented 5 years ago

I deploy my spider again and the error vanish,I think I may change some setting when I deploy it firstly.

Digenis commented 5 years ago

The bug is in your pipeline code.

Read the following guide: https://docs.python.org/2/howto/unicode.html