Passing arguments to webcrawler

[GoogleCodeExporter]

What steps will reproduce the problem?
1.create CrawlController
2.start custom WebCrawler via CrawlController
3.No way to pass arguments to WebCrawler

What is the expected output? What do you see instead?
A way to pass arguments to the WebCrawler.
The problem is that the CrawlController instantiates the WebCrawler and there
no way to pass arguments to the instance. 

What version of the product are you using? On what operating system?
Ubunto 10.4

Please provide any additional information below.
Offered solution: Pass a map of arguments to the start method and let the 
CrawlController set it in the WebCrawler instance

Original issue reported on code.google.com by homer.ho...@gmail.com on 24 Aug 2011 at 8:18

[GoogleCodeExporter]

I agree with this enhancement and will include it in the next version of 
crawler4j.

Thanks,
Yasser

Original comment by ganjisaffar@gmail.com on 24 Aug 2011 at 6:50

• Changed state: Accepted
• Added labels: Type-Enhancement
• Removed labels: Type-Defect

[GoogleCodeExporter]

Thanks for the quick reply

Attached is a patch with the offered solution above
Added a map to the WebCrawler in order to allow the CrawlController to pass 
arguments

Original comment by homer.ho...@gmail.com on 25 Aug 2011 at 9:28

Attachments:

• crawler4j.patch

[GoogleCodeExporter]

I implemented something similar here: https://github.com/cpjordan79/crawler4j

Original comment by chrisjor...@gmail.com on 22 Dec 2011 at 4:39

[GoogleCodeExporter]

As of version 3.0 this feature is implemented. See 
http://code.google.com/p/crawler4j/source/browse/src/test/java/edu/uci/ics/crawl
er4j/examples/multiple/ for an example.

-Yasser

Original comment by ganjisaffar@gmail.com on 2 Jan 2012 at 7:30

• Changed state: Done