Additional sources to evaluate

Morning fellow punchers of miscreants. You may want to consider the feeds https://www.packetmail.net/iprep_ramnode.txt and https://www.packetmail.net/iprep_CARISIRT.txt. This is the same honeypot code running on packetmail.net (206.82.85.196/30) at https://www.packetmail.net/iprep.txt but deployed on a Netherlands VPS (ramnode) and in the US (Cari). Existing parsers capable of handing 'iprep.txt' should be able to parse these two feeds without issue.

alexcpsec commented 9 years ago

@norwayfinland you guys are the best! We need to get out :hankey: together on version 0.2.0 to ingest all your new awesomeness properly.

norwayfinland commented 9 years ago

Always glad to help my friend, I'm trying to get a deployment over in LACNIC and JPNIC/APNIC land for a different demographic sampling. Hopefully this can happen sometime soon and I'll update this thread with the respective URLs. Glad I'm able to have a nominal contribution back to the greater security community itself.

alexcpsec commented 9 years ago

@norwayfinland just to confirm, this is all activity hitting a low interaction honeypot you have, right?

If so, I will file it under "inbound"

norwayfinland commented 9 years ago

"Inbound" and low interaction is a great classification. The system is completely passive and is highly opportunistic in nature.

juju4 commented 9 years ago

While comparing different tools, I'm adding the following to the list. As I was intending to add them in a pull request, I realize I might want few informations first: any way to specify list which are both inbound/outbound, how to give a confidence level, tlp or a tags ? (in a similar way than CIF do for some part)

CIF (https://github.com/csirtgadgets/massive-octo-spice/blob/develop/src/rules/default/spamhaus.yml) http://www.spamhaus.org/drop/drop.txt http://www.spamhaus.org/drop/edrop.txt http://data.phishtank.com/data/online-valid.json.gz http://s3.amazonaws.com/alexa-static/top-1m.csv.zip http://aper.svn.sourceforge.net/svnroot/aper/phishing_reply_addresses http://danger.rulez.sk/projects/bruteforceblocker/blist.php http://www.mirc.com/servers.ini http://cybercrime-tracker.net/all.php

from https://github.com/tomchop/malcom/ https://www.dan.me.uk/tornodes http://atrack.h3x.eu/api/asprox_full_csv.php http://www.malwaredomainlist.com/hostslist/mdl.xml http://malwared.malwaremustdie.org/rss.php http://www.malwaredomainlist.com/hostslist/mdl.xml

from https://github.com/jonschipp/mal-dnssearch http://labs.snort.org/feeds/ip-filter.blf http://rules.emergingthreats.net/open/suricata/rules/compromised-ips.txt http://rules.emergingthreats.net/open/suricata/rules/botcc.rules http://rules.emergingthreats.net/open/suricata/rules/tor.rules http://secure.mayhemiclabs.com/malhosts/malhosts.txt https://raw.githubusercontent.com/jonschipp/mal-dnssearch/master/mandiant_apt1.dns

from https://github.com/jpsenior/threataggregator http://www.binarydefense.com/banlist.txt https://sslbl.abuse.ch/blacklist/sslipblacklist.csv

Ponmocup (http://mcaf.ee/vw6ja) http://security-research.dyndns.org/pub/malware-feeds/ponmocup-botnet-domains.txt http://security-research.dyndns.org/pub/malware-feeds/ponmocup-botnet-ips.txt http://security-research.dyndns.org/pub/malware-feeds/ponmocup-malware-domains.txt http://security-research.dyndns.org/pub/malware-feeds/ponmocup-malware-ips.txt

norwayfinland commented 9 years ago

I would not classify or rate these threads, if you do, you present and assert a classification without fully understanding the nature of the back end. It also potentially feeds the false economy around artificial decision making without subjective and objective review of the data points. Why classify at all, couldn't the ingestion system weight accordingly? Essentially we should be presenting data and allowing the ingestion source to weight according to their criteria not making assumptions around scoring to their need. With the utmost respect I hope this makes sense my friend.

Cheers, Nathan

juju4 commented 9 years ago

Hello Nathan,

No problem on classification point, that's a design choice. Can be done later but some time suggestion helps :)

Any feedback on integration feed which are inbound+outbound? Or just do one direction and review later? Did a test and works well for spamhaus and sslbl, nok for dan tornodes https://github.com/juju4/combine/commit/01f323aafab735c2089a4b498a7863b2a85f907c

Thanks

alexcpsec commented 9 years ago

Hi, Julian. This is a great list. Thanks for doing the comparisons with other tools

You should look at the way the dev branch is organized. We have completely rewrote the way feeds are added and processed in a much more extensible "plugin" format. You will find it is way easier to add your confidence / severity scores per feed or per feed entry if they have if there.

As for the 'inbound/outbound' we are happy to review later, but as a rule of thumb, any sort of potential threat that is trying to get IN an organization (Scanning bots or machines / Spam senders) should be classified as inbound and anything that requires the organization to reach OUT to be infected (Phishing links / malware droppers / CnC hosts) should be classified as outbound.

We would really appreciate if you could work on some plugins for this entries with a pull request to our dev branch. Please review our contributing guidelines and reach out if you have any questions.

juju4 commented 9 years ago

I did some update on dev (vs my initial work on master) see https://github.com/juju4/combine/tree/dev

still work in progress

mlsecproject / combine

Additional sources to evaluate #25