search

hakluke / hakrawler

Simple, fast web crawler designed for easy, quick discovery of endpoints and assets within a web application
https://hakluke.com
GNU General Public License v3.0
4.41k stars 483 forks source link

Error parsing URL #115

Closed 0xKaran closed 2 years ago

0xKaran commented 2 years ago

I'm trying to crawl a URL from archive but it shows me an URL parse error

2021/12/30 02:12:16 Error parsing URL: parse "http://listverse.in:80/info/%3C%=urlKeyword%%3E.html?ses=Y3JlPTEzODY1ODA0OTEmdGNpZD1saXN0dmVyc2UuaW41MmE1OGEwYjljYjZlNi41MDQ4NTE0NSZma2k9NTgzMzg1ODMmdGFzaz1zZWFyY2gmZG9tYWluPWxpc3R2ZXJzZS5pbiZzPWYwZGVjYWUyNzZjMjc1NWUyYjdlJmxhbmd1YWdlPWVuJmFfaWQ9Mg==&keyword=%3C%=urlKeyword%%3E&token=%3C%=token%%3E": invalid URL escape "%=u"

How to overcome this issue?

hakluke commented 2 years ago

The URL is malformed, you need to give hakrawler URLs that are not malformed.