-
```
Add constructor to webcrawler that takes only ICrawlDecisionMaker and both
ICrawlDecisionMaker and CrawlConfiguration
```
Original issue reported on code.google.com by `sjdir...@gmail.com` on 26…
-
This project:
https://github.com/mdsol/grell
Aims at providing an easy way for ruby apps to create webcrawlers.
It will go through all the pages of some domain and you will be able to do with them w…
-
```
It doesn't reduce the depth when calling crawl recursively, so the the depth is
ineffective currently.
```
Original issue reported on code.google.com by `joelai85` on 26 Oct 2012 at 6:18
-
```
child URL missing in Printf
```
Original issue reported on code.google.com by `ort...@gmail.com` on 20 Jul 2014 at 4:16
Attachments:
- [webcrawler.go.patch](https://storage.googleapis.com/google…
-
```
It doesn't reduce the depth when calling crawl recursively, so the the depth is
ineffective currently.
```
Original issue reported on code.google.com by `joelai85` on 26 Oct 2012 at 6:18
-
```
What steps will reproduce the problem?
1.create CrawlController
2.start custom WebCrawler via CrawlController
3.No way to pass arguments to WebCrawler
What is the expected output? What do you see…
-
```
It doesn't reduce the depth when calling crawl recursively, so the the depth is
ineffective currently.
```
Original issue reported on code.google.com by `joelai85` on 26 Oct 2012 at 6:18
-
```
Make configuration like MaxThreads and UserAgentString easy from crawler
object. Maybe make a BasicWebCrawler : WebCrawler that gives this functionality.
```
Original issue reported on code.goog…
-
```
Like the stuff shown here: http://msdn.microsoft.com/en-us/library/dd997396.aspx
Not just in IThreadManagers but in the WebCrawler/ProcessPage/etc...
```
Original issue reported on code.google.c…
-
```
We can choose to not add pages that are external when config says to only crawl
internal pages. This will reduce memory and speed things up.
Patch attached.
```
Original issue reported on co…