Version used: 3.3
Request:
In order to be able to restrict crawls to a certain language, I want to inspect
the text of a page before deciding whether links should be followed.
This probably means a call to visit(page) before scheduling the links.
Is it possible to have an option for this. (I now just interchanged this in the
downloaded code).
What steps will reproduce the problem?
1.
2.
3.
What is the expected output? What do you see instead?
What version of the product are you using?
Please provide any additional information below.
Original issue reported on code.google.com by jded...@gmail.com on 24 Feb 2013 at 2:08
Original issue reported on code.google.com by
jded...@gmail.com
on 24 Feb 2013 at 2:08