A virtual host scanner that performs reverse lookups, can be used with pivot tools, detect catch-all scenarios, work around wildcards, aliases and dynamic default pages.
GNU General Public License v3.0
1.2k
stars
231
forks
source link
Detect catch-all scenarios with dynamic data and provide "likely matches" based on page differences #2
Current codebase helps to detect unique pages in catch-all scenarios by capturing the hashes of pages, and serving back the count of each hash (so you view pages which are unique).
This breaks down if something as simple as the time is served on the page causing it to be more dynamic, causing a difference in hashes. Proposal is to scrape page data using requests and work out the overlap between pages to identify the domains that are distinctly different to the others.
codingo
commented
7 years ago
Current codebase helps to detect unique pages in catch-all scenarios by capturing the hashes of pages, and serving back the count of each hash (so you view pages which are unique).
This breaks down if something as simple as the time is served on the page causing it to be more dynamic, causing a difference in hashes. Proposal is to scrape page data using requests and work out the overlap between pages to identify the domains that are distinctly different to the others.