search

jaeles-project / gospider

Gospider - Fast web spider written in Go
MIT License
2.45k stars 304 forks source link

infinite loop #73

Open dvbliw opened 2 months ago

dvbliw commented 2 months ago

When passing the gospider to a list of active hosts, there are hosts where it enters an infinite loop and is only in 1 days, apparently bringing the same thing, is there any way to avoid it?

Talanor commented 2 months ago

To add more context (maybe ?) A web server configured to always return 200 /index instead of 404 will go in a ridiculous loops of text/js/text/js/text/js/text/js/text/js/text/js/text/js for ex. I thought the -d option would mitigate that but apparently not.