Think about moving unique uri crawling check/logic to IScheduler

nhattn / abot

Automatically exported from code.google.com/p/abot

Apache License 2.0

0 stars 0 forks source link

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago

-Think about moving unique uri crawling check/logic to IScheduler so a 
DistributedScheduler would be the only change to give distributed scheduling.

Original issue reported on code.google.com by sjdir...@gmail.com on 23 Nov 2012 at 10:14

GoogleCodeExporter commented 9 years ago

unique check will stay in ICrawlDecisionMaker.ShouldCrawlPage since decision 
making should stay in this class.

Original comment by sjdir...@gmail.com on 12 Jan 2013 at 10:54