Work on IThreadManagers

[GoogleCodeExporter]

ProducerConsumerThreadManager is a resource hog if multiple instances of 
PoliteWebCrawler are created. 

ManualThreadManager uses a busy wait. Would be much faster with a spin lock or 
EventWaitHandle.

Original issue reported on code.google.com by sjdir...@gmail.com on 15 Mar 2013 at 12:16

[GoogleCodeExporter]

Original comment by sjdir...@gmail.com on 25 Mar 2013 at 9:51

• Removed labels: Milestone-Release1.1

[GoogleCodeExporter]

Also test TaskThreadManager and find its strengths/weaknesses

Original comment by sjdir...@gmail.com on 29 Mar 2013 at 5:13

[GoogleCodeExporter]

During testing of 1.1 in crawldaddy noticed that the 
ProducerConsumerThreadManager was the slowest IThreadManager impl. 
ManualThreadManager was the fastest, TaskThreadManager a close second. 

Original comment by sjdir...@gmail.com on 22 Apr 2013 at 11:03

[GoogleCodeExporter]

Add to ProducerConsumerThreadManager...

if(token.IsCancellationRequested)
  token.ThrowIfCancellationRequested();

Original comment by sjdir...@gmail.com on 26 Apr 2013 at 7:08

[GoogleCodeExporter]

//Wvtesting2
//base.CrawlAndAssert(new PoliteWebCrawler(null, null, new 
ProducerConsumerThreadManager(10), null, null, null, null, null, null));//~24 
secs
//base.CrawlAndAssert(new PoliteWebCrawler(null, null, new 
ManualThreadManager(10), null, null, null, null, null, null));//~17.90 secs
//base.CrawlAndAssert (new PoliteWebCrawler (null, null, new 
ManualNonBlockingThreadManager(10), null, null, null, null, null, 
null));//~13.88
//base.CrawlAndAssert(new PoliteWebCrawler(null, null, new 
TaskThreadManager(10), null, null, null, null, null, null));//~13.84, but more 
of a spread from 13.84 to 16

//SiteSimulator
//base.CrawlAndAssert(new PoliteWebCrawler(null, null, new 
ProducerConsumerThreadManager(10), null, null, null, null, null, null));//~7.78 
secs
//base.CrawlAndAssert(new PoliteWebCrawler(null, null, new 
ManualThreadManager(10), null, null, null, null, null, null));//~5.41 to 8 secs
//base.CrawlAndAssert (new PoliteWebCrawler (null, null, new 
ManualNonBlockingThreadManager(10), null, null, null, null, null, 
null));//~5.25 to 7.7
//base.CrawlAndAssert(new PoliteWebCrawler(null, null, new 
TaskThreadManager(10), null, null, null, null, null, null));//~5.25 to 7.78

Original comment by sjdir...@gmail.com on 27 Apr 2013 at 7:00

[GoogleCodeExporter]

Original comment by sjdir...@gmail.com on 27 Apr 2013 at 7:27

• Added labels: Milestone-Release1.2

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Added manualresetevent instead of busy wait to ManualThreadManager.

Original comment by sjdir...@gmail.com on 27 Apr 2013 at 7:45

• Changed state: Fixed