search

TheDr1ver / MISPHunter

Uses searches on 3rd party services and MISP to track actor infrastructure as it's built
Apache License 2.0
2 stars 0 forks source link

Better IOC extraction for URLs/Domains #24

Closed TheDr1ver closed 3 years ago

TheDr1ver commented 3 years ago

Also better ignore lists. E.g. this shouldn't be an extracted URL: http://tomcat.apache.org/faq/">FAQ</a apache.org should be ignored, and the url itself should stop when invalid characters start showing up.

Other ignored URLs/Domains: http://wiki.apache.org/tomcat/FrontPage http://nginx.org/">nginx.org</a>.<br/ http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd http://www.w3.org/1999/xhtml http://fonts.googleapis.com/css?family=Open+Sans:400,300

TheDr1ver commented 3 years ago

Ignore Censys/Shodan IPs found during extraction - likely relies on #12 to find the ASN of the extracted IP.

You can probably use the search results themselves to ignore these IPs. For example, Censys lists the scanning IP as services.source_ip. Shodan doesn't appear to have an equivalent at first glance, but it might be worth digging deeper into the API to see if one shows up.

TheDr1ver commented 3 years ago

Apparently this commit prevents source_ip from being stripped at the end and now it's showing up again on all the JSON blobs