dataabc
commented
2 years ago 修改settings.py，注释end_date看看。
Open luolovehk opened 2 years ago
dataabc
commented
2 years ago 修改settings.py，注释end_date看看。
luolovehk
commented
2 years ago 注释后可以实现了，
请问一下“结果文件”的路径可以修改到其他路径的吗？？ 我设置 IMAGES_STORE = 路径没效果
dataabc
commented
2 years ago 修改pipelines.py文件就可以，想修改什么文件，就改对应的。
luolovehk
commented
2 years ago 修改pipelines.py文件就可以，想修改什么文件，就改对应的。
好的，感谢解答
luolovehk
commented
2 years ago 每天学点摄影技巧
搜索这个关键字报错
设置了 要搜索的微博类型，爬全部报错，设置1爬原创就不会报错
2022-05-09 15:38:22 [scrapy.core.scraper] ERROR: Spider error processing <GET ht tps://s.weibo.com/weibo?q=%E6%AF%8F%E5%A4%A9%E5%AD%A6%E7%82%B9%E6%91%84%E5%BD%B1 %E6%8A%80%E5%B7%A7&typeall=1&suball=1×cope=custom:2022-03-01-0:2022-05-10-0
(referer: None) Traceback (most recent call last): File "d:\python\python37\lib\site-packages\scrapy\utils\defer.py", line 132, i n iter_errback yield next(it) File "d:\python\python37\lib\site-packages\scrapy\utils\python.py", line 354, in next return next(self.data) File "d:\python\python37\lib\site-packages\scrapy\utils\python.py", line 354, in next return next(self.data) File "d:\python\python37\lib\site-packages\scrapy\core\spidermw.py", line 66, in _evaluate_iterable for r in iterable: File "d:\python\python37\lib\site-packages\scrapy\spidermiddlewares\offsite.py ", line 29, in process_spider_output for x in result: File "d:\python\python37\lib\site-packages\scrapy\core\spidermw.py", line 66, in _evaluate_iterable for r in iterable: File "d:\python\python37\lib\site-packages\scrapy\spidermiddlewares\referer.py ", line 342, in
return (_set_referer(r) for r in result or ()) File "d:\python\python37\lib\site-packages\scrapy\core\spidermw.py", line 66, in _evaluate_iterable for r in iterable: File "d:\python\python37\lib\site-packages\scrapy\spidermiddlewares\urllength. py", line 40, in return (r for r in result or () if _filter(r)) File "d:\python\python37\lib\site-packages\scrapy\core\spidermw.py", line 66, in _evaluate_iterable for r in iterable: File "d:\python\python37\lib\site-packages\scrapy\spidermiddlewares\depth.py", line 58, in return (r for r in result or () if _filter(r)) File "d:\python\python37\lib\site-packages\scrapy\core\spidermw.py", line 66, in _evaluate_iterable for r in iterable: File "D:\weibo-search-master\weibo\spiders\search.py", line 107, in parse for weibo in self.parse_weibo(response): File "D:\weibo-search-master\weibo\spiders\search.py", line 464, in parse_weib o ).extract_first()[4:] TypeError: 'NoneType' object is not subscriptable
dataabc
commented
2 years ago 是不是没有设置cookie？
可以设置爬取当天到某一天的结果吗？通过定时执行。爬取最新的内容