search

gauthamsunjay / ariados

A scalable web crawling framework using lambdas
0 stars 0 forks source link

adding stopping condition for threads in master #24

Closed gauthamsunjay closed 5 years ago

gauthamsunjay commented 5 years ago

If we are still processing links, we might in future encounter another URL from that domain right?