只能抓取100页的数据

[tonray]

Progress: 32%|█████████████████████████▍ | 100/314 [31:55<1:08:18, 19.15s/it] 'NoneType' object has no attribute 'xpath' Traceback (most recent call last): File "/root/weiboSpider/weibo_spider/spider.py", line 169, in get_weibo_info self.user_config, page, self.filter).get_one_page( File "/root/weiboSpider/weibo_spider/parser/page_parser.py", line 45, in init info = self.selector.xpath("//div[@class='c']") AttributeError: 'NoneType' object has no attribute 'xpath' 共爬取997条原创微博 信息抓取完毕

---

运行到32%，100页的位置，就中断了

[tonray]

又尝试了下，这次抓的更少了

Progress: 12%|█████████▌ | 37/315 [09:12<1:09:13, 14.94s/it] 'NoneType' object has no attribute 'xpath' Traceback (most recent call last): File "/root/weiboSpider/weibo_spider/spider.py", line 169, in get_weibo_info self.user_config, page, self.filter).get_one_page( File "/root/weiboSpider/weibo_spider/parser/page_parser.py", line 45, in init info = self.selector.xpath("//div[@class='c']") AttributeError: 'NoneType' object has no attribute 'xpath' 共爬取371条原创微博 信息抓取完毕

[dataabc]

感谢反馈。

应该是速度太快了，参考常见问题修改即可。

[tonray]

谢谢

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

Closing as stale, please reopen if you'd like to work on this further.