taganaka / polipus

Polipus: distributed and scalable web-crawler framework
MIT License
92 stars 32 forks source link

Check for invalid uri in redirects #64

Closed pcboy closed 9 years ago

pcboy commented 9 years ago

Got the issue on a few websites. The redirect uri wasn't even an uri so everything crashed.