search

tomnomnom / httprobe

Take a list of domains and probe for working HTTP and HTTPS servers
MIT License
2.84k stars 500 forks source link

same results #2

Open Phoenix1112 opened 5 years ago

Phoenix1112 commented 5 years ago

hello. thanks for this project.. i have subdomain list.. It has 134 subdomain names. When I get the protocol output with httprobe, it gets 264... adds the same subdomain name to https and http 2 times.

example my subdomain list:

videos.staging-airtableblocks.com staging.staging-airtableblocks.com

httprobe output:

http://videos.staging-airtableblocks.com https://videos.staging-airtableblocks.com http://staging.staging-airtableblocks.com https://staging.staging-airtableblocks.com

first he needs to check the https protocol. if the https protocol is used, it should add the https protocol subdomain name to the output file. If a site is added to the output file with the https protocol, the http protocol should not be added... I'll be happy if you solve this problem quickly.

tomnomnom commented 5 years ago

Hi! Thanks for raising an issue! :)

I'm afraid this is 100% intended behaviour and I don't consider it to be a problem.

Web servers often behave differently on HTTP and HTTPS; especially when looking at - for example - open redirect issues so I want both in my results.

I think it's worth pointing out that HTTP on port 80 and HTTPS on port 443 are only the default probes (and they can even be disabled with the -s flag), and the kind of behaviour you describe wouldn't map well when there are additional probes in use, especially where the default probes are being skipped.

Sorry the tool doesn't quite do what you want it's current form. We could perhaps look at adding a flag to optionally enable the behaviour you suggest, but I don't know how that would interact with additional probes etc. I'm open to ideas on that front though!

Phoenix1112 commented 5 years ago

thank you for replying.. sometimes we have long subdomain lists in our hands... for example, some subdomains do not work with the http protocol.so changing a protocol when checking a subdomain can be really tedious and long. what really matters here is not which ports of the site are active or not. It is more important to find out which protocol the website works with...

Suppose we want to take screenshots of sites that are in a subdomain list. The sites in the subdomain list must include the http or https protocol at the beginning. The httprobe program parses the same sites with both the http and https protocol. After taking a screenshot of a site running with the https protocol, it would be a waste of time to open the same site with the http protocol and try to take a screenshot... also, we cannot open a site that works with https with the http protocol. this may cause certificate errors and result in unwanted screen shots.

If a site works with which protocol, you must provide that site as output... if the site is opened with the https protocol, you should not reflect the http protocol to the output. maybe this may not be related to the httprobe program. however, you can start this feature with another project if you want.

you can also add even more advanced features...