asepaprianto / crawler4j

Automatically exported from code.google.com/p/crawler4j
0 stars 0 forks source link

Waiting 30sec before cleaning everything #313

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago
What steps will reproduce the problem?
1.  Crawler4j always wait for 30 sec before.we can see that in the follwoing 
output!! can't we change that 

INFO [Thread-2] It looks like no thread is working, waiting for 10 seconds to 
make sure... INFO [Thread-1] It looks like no thread is working, waiting for 10 
seconds to make sure... INFO [Thread-2] No thread is working and no more URLs 
are in queue waiting for another 10 seconds to make sure... INFO [Thread-1] No 
thread is working and no more URLs are in queue waiting for another 10 seconds 
to make sure... INFO [Thread-2] All of the crawlers are stopped. Finishing the 
process... INFO [Thread-2] Waiting for 10 seconds before final clean up... INFO 
[Thread-1] All of the crawlers are stopped. Finishing the process... INFO 
[Thread-1] Waiting for 10 seconds before final clean up... 

What is the expected output? What do you see instead?
It shoul not wait for those 30 sec. Or at least reduce 30sec 5sec etc.

What version of the product are you using?
3.0

Please provide any additional information below.

Original issue reported on code.google.com by yenumula...@gmail.com on 29 Sep 2014 at 8:33

GoogleCodeExporter commented 9 years ago
We won't change that code architecture for now.

But you can change it in your own code in the CrawlController.java

Every sleep(10) you can change to sleep(3) or any other number of seconds you 
want.

Original comment by avrah...@gmail.com on 29 Sep 2014 at 1:47