search

Aqua-Dream / Tieba_Spider

百度贴吧爬虫(基于scrapy和mysql)
404 stars 116 forks source link

爬取时出现了AttributeError: 'list' object has no attribute 'values'. #12

Closed Nekofish-L closed 5 years ago

Nekofish-L commented 5 years ago

错误信息如下： File "C:\ProgramData\Anaconda3\lib\site-packages\scrapy\utils\defer.py", line 102, in iter_errback yield next(it) File "C:\ProgramData\Anaconda3\lib\site-packages\scrapy\spidermiddlewares\offsite.py", line 30, in process_spider_output for x in result: File "C:\ProgramData\Anaconda3\lib\site-packages\scrapy\spidermiddlewares\referer.py", line 339, in return (_set_referer(r) for r in result or ()) File "C:\ProgramData\Anaconda3\lib\site-packages\scrapy\spidermiddlewares\urllength.py", line 37, in return (r for r in result or () if _filter(r)) File "C:\ProgramData\Anaconda3\lib\site-packages\scrapy\spidermiddlewares\depth.py", line 58, in return (r for r in result or () if _filter(r)) File "C:\Users\surface\Desktop\Tieba_Spider\tieba\spiders\tieba_spider.py", line 82, in parse_comment for value in comment_list.values(): AttributeError: 'list' object has no attribute 'values'

LCG22 commented 5 years ago

这是因为有的页面没爬取到数据，你要自己做一个判断

Aqua-Dream commented 5 years ago

能否提供下这是爬哪个帖子时出现的问题？

Nekofish-L commented 5 years ago

能否提供下这是爬哪个帖子时出现的问题？

在爬取民科吧的时候出现的问题，当时没注意是哪个帖子的问题

Aqua-Dream commented 5 years ago

下次出现问题的时候麻烦提供一下，多谢啦