All the seeds get crawled or visited before any further depth is crawled

sujit-kr / crawler4j

Automatically exported from code.google.com/p/crawler4j

0 stars 0 forks source link

All the seeds get crawled or visited before any further depth is crawled #37

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago

What steps will reproduce the problem?

Add a good number of seeds, say 100,000.

What is the expected output? What do you see instead?

Crawler4j is waiting to finish up all the seeds before it even start going down 
through any single seed. I was expecting it to start the deeper depths also in 
parallel

What version of the product are you using? On what operating system?
2.6
Windows 7

Original issue reported on code.google.com by aravin...@gmail.com on 2 May 2011 at 11:20

GoogleCodeExporter commented 9 years ago

Original comment by ganjisaffar@gmail.com on 2 May 2011 at 11:57

Changed state: Invalid