search

0xd34db33f / gfyp

Unification of dnstwist + SQLite + Email reporting. Set it as a cron job that runs every hour, give it a list of domains and email addresses for reporting, then watch it go find stuff.
66 stars 13 forks source link

feat: Poke at found domain variant with web requests #13

Open kristovatlas opened 7 years ago

kristovatlas commented 7 years ago

Most domain variants in my experience are boring squatted domains, and could be easily differentiated with a simple HTTP request to http://example.com/.

This could be further extended to support customization by the user to look for the presence or absence of certain tell-tale strings in webpages and for other URL patterns, e.g. to detect commonly known phishing kit installations.

kristovatlas commented 7 years ago

Another reason why this would be a useful feature based on my recent observations, @0xd34db33f: Phishing sites sometimes start out appearing as squatted domains and later change into phishing sites later on without changing DNS settings. It would be cool if this tool could optionally monitor sites over time and alert to changes aside from purely DNS-based ones.

0xd34db33f commented 7 years ago

Love the idea, should be pretty trivial using requests library, md5'ing the downloaded information, and then checking if it's different than the value in a new table we setup. Only tricky part is I've seen a few typo-squatters at $dayjob where they are filled with dynamic content. Wonder if we tweak it so util.py can have an option to disable it if the domain is problematic?

0xd34db33f commented 6 years ago

Started coding this up using Scrapy a little bit ago. Think I've got something, so I'll take ownership for now.