search

opensangja / abot

Automatically exported from code.google.com/p/abot
Apache License 2.0
0 stars 0 forks source link

Crawler crawls over MaxPagesToCrawl by up to X pages. X being the number of MaxConcurrentThreads #58

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
What steps will reproduce the problem?
1.Run Abot.Console
2.Crawl http://wvtesting2.com/

What is the expected output? 
Configured to crawl 10 pages and should stop crawling after 10 pages.

What do you see instead?
Crawls 18 to 19.

Also need to check why the integration test passes when MaxPagesToCrawl is set 
to 10. I assume its related to network latency vs localhost site.

Original issue reported on code.google.com by sjdir...@gmail.com on 4 Jan 2013 at 12:51

GoogleCodeExporter commented 9 years ago

Original comment by sjdir...@gmail.com on 4 Jan 2013 at 6:07