search

Python3WebSpider / ScrapyRedisBloomFilter

Scrapy Redis Bloom Filter
173 stars 52 forks source link

有个弊端把start_urls里面的链接也给去重了 #9

Open dahuangfeng123 opened 2 years ago

dahuangfeng123 commented 2 years ago

起始链接被去重后面就没法抓取了，望改进