search

tomnomnom / httprobe

Take a list of domains and probe for working HTTP and HTTPS servers
MIT License
2.84k stars 500 forks source link

False Positive #35

Open Naategh opened 4 years ago

Naategh commented 4 years ago

Hi! I don't know why but it returns wrong results sometimes:

$ echo "genrih.resizer.mail.ru" | httprobe
https://genrih.resizer.mail.ru
http://genrih.resizer.mail.ru
gbiagomba commented 3 years ago

@Naategh it might be just appending both http/s by default without checking the domain? have you tried passing explicit flags like specifying a port and seeing if that yields better results?

Naategh commented 3 years ago

@gbiagomba thanks for your note. BTW I tried this and yet it returns wrong results.

gbiagomba commented 3 years ago

Hmmm I tried doing an nslookup and I didn't get a response. I wonder if you can run it and see the "traffic" coming out of it. Also it makes me wonder if it can't resolve a host does it just go into dumb mode and append http and https

BorrowedBastion commented 3 years ago

This can happen when the domain is using a wildcard subdomain redirect. Most of the time, it will redirect to a non-existent subdomain to the top-level domain's 404 page.

Since this functionality is pretty common, it may be good to add in a test for non-existent subdomains returning a response (ex, "probablynotreal12345678.yourdomain.com"), to notify the user that there will be false positives.

I'm not an expert with Go, but if it'd be helpful I'd be happy to do a pull request to add that functionality.

Back to the original question from last year. When I run $ echo "genrih.resizer.mail.ru" | httprobe it no longer returns a false positive. I think that is due to mail.ru changing a redirect policy on the second tier of subdomains. It will still return false positives for .mail.ru, just not .example.mail.ru.