HurricaneLabs / machinae

Machinae Security Intelligence Collector
MIT License
504 stars 101 forks source link

Add Cymon.io #11

Closed exp0se closed 8 years ago

exp0se commented 8 years ago

Found your tool, i was actually started to write a similar tool myself here. Since your tool have much more sources, i think i am going to use your tool instead!

Please integrate https://cymon.io/

Your could copy my code if you want.

mcm commented 8 years ago

@exp0se - Thanks for the suggestion. I will work on getting this added!

mcm commented 8 years ago

All done. Implemented in 1.3.0 (e0c274ceaa775ec94eb1cb10cccbd08be201f253), a few new features to support their API, and machinae.yml updated with support for each of their API components.

Let me know if it works for you. Thanks again for the suggestion.

exp0se commented 8 years ago

Thanks!

It works, but there is a small problem with URL:

[!] Error from Cymon.io URL Lookup: 500 Server Error: INTERNAL SERVER ERROR

This is a problem when you try to analyze URL without http or https prefix. Maybe append http:// if otype = URL?

mcm commented 8 years ago

@exp0se are you manually telling machinae that it's a URL, like so:

~/g/h/machinae ❯❯❯ machinae --detect-otype -O url faker.su/data/entry/steam/Steam.exe faker.su/data/entry/steam/Steam.exe: url ~/g/h/machinae ❯❯❯

Because without that, it detects it as an fqdn:

~/g/h/machinae ❯❯❯ machinae --detect-otype faker.su/data/entry/steam/Steam.exe faker.su/data/entry/steam/Steam.exe: fqdn ~/g/h/machinae ❯❯❯

That said, I think I can make it prepend http:// if the otype is URL, whether auto-detected or forced.

mcm commented 8 years ago

@exp0se Fixed in 1.3.1, please give it a shot and let me know

exp0se commented 8 years ago

Yeah, i manually set this as URL

I think there is a problem with urlencoding. I noticed that Cymon is kinda picky about that, when i was testing API myself.

Anyway, here is what on 1.3.1

~$ machinae -O url wonderph.com/dbsys.php [.] Requesting https://www.virustotal.com/vtapi/v2/url/report?apikey=308211ef74a1044ea98134424b3d20769451d25beda0b808a8b61036badc0ea1&resource=wonderph.com%2Fdbsys.php (GET) [.] Requesting https://cymon.io/api/nexus/v1/url/wonderph.com%252Fdbsys.php (GET) [.] Requesting https://www.fortiguard.com/ip_rep/index.php?data=wonderph.com/dbsys.php&lookup=Lookup (GET) [.] Requesting http://www.toolsvoid.com/unshorten-url (POST)


[+] VirusTotal URL Report results [-] Date submitted: 2016-03-01 18:21:36 [-] Detected scanners: 12 [-] Total scanners: 68 [-] URL Scanner: ('BitDefender', 'malware site') [-] URL Scanner: ('Malware Domain Blocklist', 'malicious site') [-] URL Scanner: ('MalwareDomainList', 'malicious site') [-] URL Scanner: ('Sophos', 'malicious site') [-] URL Scanner: ('Trustwave', 'malicious site') [-] URL Scanner: ('AutoShun', 'malicious site') [-] URL Scanner: ('G-Data', 'malware site') [-] URL Scanner: ('ESET', 'malware site') [-] URL Scanner: ('Sangfor', 'malware site') [-] URL Scanner: ('Websense ThreatSeeker', 'malicious site') [-] URL Scanner: ('Fortinet', 'malware site') [-] URL Scanner: ('CRDF', 'malicious site') [!] Error from Cymon.io URL Lookup: 500 Server Error: INTERNAL SERVER ERROR for url: https://cymon.io/api/nexus/v1/url/wonderph.com%252Fdbsys.php [+] Fortinet Category results [-] Fortinet URL Category: Malicious Websites [-] No URL Unshorten results

mcm commented 8 years ago

Are you sure you updated to 1.3.1? Running that same thing I get:

` [.] Requesting http://www.toolsvoid.com/unshorten-url (POST) [.] Requesting https://www.virustotal.com/vtapi/v2/url/report?apikey=308211ef74a1044ea98134424b3d20769451d25beda0b808a8b61036badc0ea1&resource=http%3A%2F%2Fwonderph.com%2Fdbsys.php (GET) [.] Requesting https://cymon.io/api/nexus/v1/url/http%253A%252F%252Fwonderph.com%252Fdbsys.php (GET)


[-] No URL Unshorten results [+] VirusTotal URL Report results [-] Date submitted: 2016-03-01 18:21:36 [-] Detected scanners: 12 [-] Total scanners: 68 [-] URL Scanner: ('Sophos', 'malicious site') [-] URL Scanner: ('MalwareDomainList', 'malicious site') [-] URL Scanner: ('Fortinet', 'malware site') [-] URL Scanner: ('Websense ThreatSeeker', 'malicious site') [-] URL Scanner: ('ESET', 'malware site') [-] URL Scanner: ('Malware Domain Blocklist', 'malicious site') [-] URL Scanner: ('BitDefender', 'malware site') [-] URL Scanner: ('G-Data', 'malware site') [-] URL Scanner: ('Trustwave', 'malicious site') [-] URL Scanner: ('CRDF', 'malicious site') [-] URL Scanner: ('Sangfor', 'malware site') [-] URL Scanner: ('AutoShun', 'malicious site') [+] Cymon.io URL Lookup results [-] URL listed by: urlquery.net [-] Associated IP: 160.153.54.66 ~ ❯❯❯ `

exp0se commented 8 years ago

Try without http://

I am able to reproduce without http://, but if you pass http:// first - it works.

mcm commented 8 years ago

@exp0se It's showing the http:// because it's adding it automatically now. With a fresh install of machinae 1.3.1 on a new system that's never had machinae installed before:

steve@covalence:~$ machinae -O url wonderph.com/dbsys.php
Warning: operating without a config file. This is probably not what you want. To correct this, fetch a copy of the default configuration file from https://github.com/hurricanelabs/machinae and place it in /etc/machinae.yml or ~/.machinae.yml and run again.
********************************************************************************
* Information for http://wonderph.com/dbsys.php
* Observable type: url (Auto-detected: False)
********************************************************************************

steve@covalence:~$