search

hellock / icrawler

A multi-thread crawler framework with many builtin image crawlers provided.
http://icrawler.readthedocs.io/en/latest/
MIT License
857 stars 174 forks source link

add option to skip download for existing files #48

Closed dreamflasher closed 6 years ago

dreamflasher commented 6 years ago

solves https://github.com/hellock/icrawler/issues/47

hellock commented 6 years ago

Thanks for your work and sorry for the late feedback. I just add a new method exists() to the storage class and make some changes accordingly.