search

mohankreddy / crawler4j

Automatically exported from code.google.com/p/crawler4j
0 stars 0 forks source link

how to incremental crawler a site? #29

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 

every time when i start a crawler, it delete then "frontier" directory, and 
begin from the first page,
how to download a site from last stopped place? please help.

Original issue reported on code.google.com by wanxiang.xing@gmail.com on 24 Mar 2011 at 6:43

GoogleCodeExporter commented 9 years ago 
http://code.google.com/p/crawler4j/issues/detail?id=17
sorry for the repeated quiz.

Original comment by wanxiang.xing@gmail.com on 24 Mar 2011 at 6:45

GoogleCodeExporter commented 9 years ago 
I have committed the new version of the source code which supports this 
feature. You can checkout it from svn. I will add the jar file to downloads 
section in a few days once it's fully tested.

-Yasser

Original comment by ganjisaffar@gmail.com on 29 Mar 2011 at 2:41