search

mohankreddy / crawler4j

Automatically exported from code.google.com/p/crawler4j
0 stars 0 forks source link

How do we dynamically set the websites that the crawler should visit #8

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
We start with a seed site and then we want to restrict the crawler to only 
follow links that are specified dynamically. We'd like to be able to pass 
that list of links to WebCrawler via a method such as setAllowedHostnames() 
since we don't want to hard code the list of sites in shouldVisit(). What's 
the best way to accomplish this?

Original issue reported on code.google.com by eduf...@gmail.com on 6 May 2010 at 1:37

GoogleCodeExporter commented 9 years ago 
This is typical Java coding! The answer depends on you. You can call any 
function
from shouldVisit.

Original comment by ganjisaffar@gmail.com on 6 May 2010 at 3:05