search

cif2cif / ldspider

Automatically exported from code.google.com/p/ldspider
0 stars 0 forks source link

Check whether large pld queues are access more often than small pld queues #20

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
Load seed list with one large pld and many smaller ones.

Make sure that the large pld queue gets hit from the beginning (so that the 
large pld access is spread out during the crawl)

Original issue reported on code.google.com by andr...@harth.org on 6 May 2012 at 2:07

GoogleCodeExporter commented 9 years ago 
For the breadth-first queues, this should be the case.

Original comment by tob.kae...@gmail.com on 23 Jun 2014 at 3:44