MISP2memcached
Load IOCs from MISP Threat sharing platform to memcached. Use Logstash to enrich your SIEM logs.
Examples are provided for Elasticsearch based SIEM with ECS mapping, but you can modify the examples to work with your setup.
Check out the Getting started if you want to see a simple use case on how to enrich data from Sysmon shipped by Winlogbeat.
Requirements
- pymemcahe
- requests
- PyYAML
Installation
After configuration simply run misp2memcahed.py as a cron job.
Configuration
Basic settings for basic memcached setup. Clusters are not currently supported.
memcached:
host: 127.0.0.1
port: 11211MISP configuration. Set ignore_cert_errors to true if working with self-signed certificate. Don't use for production.
When MISP2memcached fails to find misp-stats key in memcached or misp-stats is 0 initial_event_timestamp value is used to do the initial
load of events from MISP. This will load all events that were modified in the specified timeframe.
On normal run when key misp-stats is present and is not 0 refresh_event_timestamp value is used to only load events modified in
that timeframe. You should configure this value based on your configuration of fetch_feeds or pull_all scheduled tasks in MISP.
misp:
url: https://misp.localhost
token: "fA3rESKxzAH2XfUyHAfRDTNcovlFDYQd4mc4kdCf"
ignore_cert_errors: false
initial_event_timestamp: 356d
refresh_event_timestamp: 1hIOCs are split into three arbitrary categories:
- hash (misp-md5, misp-sha1, misp-sha256, misp-sha512, misp-imphash)
- network (misp-ip, misp-domain)
- web (misp-url)
If the enabled variable is set to true data will be queried in MISP and loaded into memcached. Expires sets the time in seconds when the key is removed from memcached. 0 sets the keys to never expire.
hash:
enabled: true
expires: 0Example configuration
memcached:
host: 127.0.0.1
port: 11211
misp:
url: https://misp.local
token: "fA3rESKxzAH2XfUyHAfRDTNcovlFDYQd4mc4kdCf"
ignore_cert_errors: false
initial_event_timestamp: 30d
refresh_event_timestamp: 5m
hash:
enabled: true
expires: 0
network:
enabled: true
expires: 0
web:
enabled: true
expires: 0Supported IOCs
| Memcached namespace | MISP types | Comment |
|---|---|---|
| misp-md5 | md5,filename|md5 | Filename is ignored |
| misp-sha1 | sha1,filename|sha1 | Filename is ignored |
| misp-sha256 | sha256, filename|sha256 | Filename is ignored |
| misp-sha512 | sha512, filename|sha512 | Filename is ignored |
| misp-imphash | imphash, filename|imphash | Filename is ignored |
| misp-ip | ip-dst, ip-src, ip-dst|port, ip-src|port, domain|ip | Port is ignored, Domain goes into misp-domain |
| misp-domain | domain, domain|ip | IP goes into misp-ip |
| misp-url | url |
Known problems
- Memcached doesn't allow whitespace characters in keys. This makes it unusable for registry keys, user agents etc. without adding some form of encoding
- Memcached TLS and authentication support is kind of unintuitive and a pain to configure. Redis would be much better fit, unfortunately there is no redis filter plugin for Logstash.
- Must use custom ruby filter to parse results from memcached in Logstash, this is not ideal, but unavoidable. Again redis would probably be a better fit.
- Probably many other
Based on
This tool was inspired by