Currently, while crawling, the crawler parses all links from every html link,
then uses those links as seeds.
But when encountering Binary or plain text (text/plain) files those links
aren't parsed and retrieved as seeds.
Upgrade the crawler to parse links from binary and text files.
Original issue reported on code.google.com by avrah...@gmail.com on 23 Sep 2014 at 11:08
Original issue reported on code.google.com by
avrah...@gmail.com
on 23 Sep 2014 at 11:08