However, the from_spider ONLY implements in scrapy_redis.dupefilter.RFPDupeFilter, while scrapy.dupefilters.BaseDupeFilter not declares. Which will raise
File "D:\Anaconda\anaconda3\envs\scrapy\Lib\site-packages\scrapy\crawler.py", line 160, in crawl
yield self.engine.open_spider(self.spider, start_requests)
AttributeError: type object 'RFPDupeFilter' has no attribute 'from_spider'
and
File "D:\Anaconda\anaconda3\envs\scrapy\Lib\site-packages\scrapy_redis\scheduler.py", line 149, in flush
self.df.clear()
AttributeError: 'Scheduler' object has no attribute 'df'
https://github.com/rmax/scrapy-redis/blob/48a7a8921ae064fe7b4202b130f1054ede9103d6/src/scrapy_redis/scheduler.py#L136
calls
from_spider
in a dupefilter class.However, the
from_spider
ONLY implements inscrapy_redis.dupefilter.RFPDupeFilter
, whilescrapy.dupefilters.BaseDupeFilter
not declares. Which will raiseand
Another user also met the same question: https://github.com/rmax/scrapy-redis/issues/242#issuecomment-2154526694