search

TurboWay / spiderman

基于 scrapy-redis 的通用分布式爬虫框架
MIT License
584 stars 128 forks source link

关于框架的两个警告 #41

Open donneyluck opened 1 year ago

donneyluck commented 1 year ago

1 启动时 [py.warnings] WARNING: /home/donney/.local/lib/python3.10/site-packages/scrapy/utils/request.py:232: ScrapyDeprecationWarning: '2.6' is a deprecated value for the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting.

It is also the default value. In other words, it is normal to get this warning if you have not defined a value for the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting. This is so for backward compatibility reasons, but it will change in a future version of Scrapy.

See the documentation of the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting for information on how to handle this deprecation. return cls(crawler)

2.运行中 2023-02-17 15:30:32 [py.warnings] WARNING: /home/donney/.local/lib/python3.10/site-packages/scrapy_redis/spiders.py:197: ScrapyDeprecationWarning: Passing a 'spider' argument to ExecutionEngine.crawl is deprecated self.crawler.engine.crawl(req, spider=self)

麻烦大佬看一下 尝试修没有好

TurboWay commented 1 year ago

看起来是scrapy、scrapy_redis的版本问题，可以尝试降级，或者注释源码中的以上warnings