search

jungjonghun / crawler4j

Automatically exported from code.google.com/p/crawler4j
0 stars 0 forks source link

File URLs Fetching #110

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
Hi Yasser,

It would be good if fetching of file URLs is supported as well. File URLs stay 
as links in some web sites. They might be fetched to check if they are broken 
or not to find broken ones.

Regs

Original issue reported on code.google.com by mansur.u...@gmail.com on 20 Jan 2012 at 9:05

GoogleCodeExporter commented 9 years ago 
+1

You mean "file://" ?? Yes it makes sense to check for broken links. Btw, we can 
easily confirm just by checking the protocol startsWith "file://" (to avoid 
fetching) and add it to broken links map.

Original comment by w3engine...@gmail.com on 22 Jan 2012 at 6:46

GoogleCodeExporter commented 9 years ago

Original comment by ganjisaffar@gmail.com on 22 Jan 2012 at 8:15