search

smasher125354 / crawler4j

Automatically exported from code.google.com/p/crawler4j
0 stars 0 forks source link

Crawler is slow #339

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
Hi,

The speed we get when using crawler4j is about 5 links per second when we set 
number of crawlers to 50.

and if we increase the number of crawler even for 500 or 2000, we still get the 
same result.

is there any suggestion how could we increase the speed?

Original issue reported on code.google.com by khalid.t...@gmail.com on 8 Mar 2015 at 10:00