search

nikhilgeit / abot

Automatically exported from code.google.com/p/abot
Apache License 2.0
0 stars 0 forks source link

Update documentation to reflect 1.1 changes #57

Closed GoogleCodeExporter closed 8 years ago

GoogleCodeExporter commented 8 years ago 
-New WebCrawler constructor args (IDomainLimiter)
-New WebCrawler constructor with only ICrawlDecisionMaker
-New WebCrawler constructor with both ICrawlDecisionMaker and CrawlConfiguration
-Add MaxTimeToCrawl config item
-Add MaxPagesToCrawlPerDomain config item
-All assemblies now target .NET framework 4.0
-See comments for a running list as they are implemented

Original issue reported on code.google.com by sjdir...@gmail.com on 30 Dec 2012 at 6:46

GoogleCodeExporter commented 8 years ago 
_shouldDownloadPageContentDecisionMaker
_shouldDownloadPageDecisionMaker
_shouldDownloadPageLinksDecisionMaker

Original comment by sjdir...@gmail.com on 2 Jan 2013 at 7:15

GoogleCodeExporter commented 8 years ago 
-Demo no longer needs http://localhost, use user prompt instead

Original comment by sjdir...@gmail.com on 3 Jan 2013 at 11:13

GoogleCodeExporter commented 8 years ago 
[deleted comment]
GoogleCodeExporter commented 8 years ago 
_isInternalUriDecisionMaker

Original comment by sjdir...@gmail.com on 30 Jan 2013 at 5:01

GoogleCodeExporter commented 8 years ago 
-no more ilmerge
-passing htmldocument as part of CRawledPage

Original comment by sjdir...@gmail.com on 1 Feb 2013 at 7:53

GoogleCodeExporter commented 8 years ago 
IHyperlinkParser now uses GetLinks(CrawledPage)

Original comment by sjdir...@gmail.com on 4 Feb 2013 at 5:24

GoogleCodeExporter commented 8 years ago 
-HtmlDocument and CQ available in crawl pipeline 
-New config value ShouldLoadHtmlAgilityPack
-New config value ShouldLoadCsQuery

Original comment by sjdir...@gmail.com on 4 Feb 2013 at 9:05

GoogleCodeExporter commented 8 years ago 
CrawlBag and New new way to edit crawlconfiguration after loading it from file 
if available.

Original comment by sjdir...@gmail.com on 11 Feb 2013 at 12:29

GoogleCodeExporter commented 8 years ago 
Add new memorymonitor, new pagerequester options, new producer consumer stuff.

Original comment by sjdir...@gmail.com on 12 Mar 2013 at 10:32

GoogleCodeExporter commented 8 years ago 
Be sure to have a full section explaining how to use the 
memorymanager/monitors. Ie.. they must set maxMemoryUsageCachetimeInSeconds to 
a value greater than 1 when maxMemoryUsageInMb is greater than zero.

Original comment by sjdir...@gmail.com on 13 Mar 2013 at 7:30

GoogleCodeExporter commented 8 years ago

Original comment by sjdir...@gmail.com on 29 Mar 2013 at 5:21

GoogleCodeExporter commented 8 years ago

Original comment by sjdir...@gmail.com on 9 Apr 2013 at 5:26