hakluke / hakrawler

Simple, fast web crawler designed for easy, quick discovery of endpoints and assets within a web application
https://hakluke.com
GNU General Public License v3.0
4.41k stars 483 forks source link

Add `-i` flag (crawl inside path) and display an error on stderr if improper URLs are being given #130

Closed garlic0x1 closed 1 year ago

garlic0x1 commented 2 years ago

url.Parse does not return an error most of the time because it treats random strings as relative URLs.

garlic0x1 commented 2 years ago

Just pushed another update that adds the -i flag for limiting the crawl to inside a path.
for example echo https://example.com/outer_path/path1 | hakrawler -i will not crawl https://example.com/outer_path/

hakluke commented 1 year ago

This is really great work, thanks garlic0x1 🙂