search

gauthamsunjay / ariados

A scalable web crawling framework using lambdas
0 stars 0 forks source link

Caching URLs #13

Open cqhfut opened 5 years ago

cqhfut commented 5 years ago

Fetched data is cached in cases where the parsing logic changes but the site data is the same.