search

TeamHG-Memex / scrapy-crawl-once

Scrapy middleware which allows to crawl only new content
MIT License
79 stars 23 forks source link

Alternative to DELTAFETCH_RESET not implemented #4

Open SteveSmirnoff opened 3 years ago

SteveSmirnoff commented 3 years ago

scrapy-crawl-once has no built-in way of clearing out all seen requests via settings.