Closed GoogleCodeExporter closed 9 years ago
Make configuration like MaxThreads and UserAgentString easy from crawler object. Maybe make a BasicWebCrawler : WebCrawler that gives this functionality.
Original issue reported on code.google.com by sjdir...@gmail.com on 14 Oct 2012 at 9:29
sjdir...@gmail.com
Original comment by sjdir...@gmail.com on 14 Oct 2012 at 11:17
already fixed in 1.0
Original comment by sjdir...@gmail.com on 23 Nov 2012 at 11:52
Original issue reported on code.google.com by
sjdir...@gmail.com
on 14 Oct 2012 at 9:29