Move to ELK-Stack 5.0 - Githubissues

telekom-security / tpotce

🍯 T-Pot - The All In One Multi Honeypot Platform 🐝

GNU General Public License v3.0

6.32k stars 1.03k forks source link

Move to ELK-Stack 5.0 #22

Closed t3chn0m4g3 closed 6 years ago

t3chn0m4g3 commented 8 years ago

[x] build basic container with elk 5.x
[x] add CI / logos
[x] no head plugin, get head standalone working
[x] set shards / replicas
[x] solution for pre-building index
[x] rebuild visualizations
[x] rebuild dashboards
[x] dashboards, visualizations, searches need some finetuning
[x] limit ram / cpu usage
[x] fix curator
[x] set recommended sysctl.conf on host / for container start
[x] maxmind released new GeoIP v2 ASN database, integration needed! (http://dev.maxmind.com/geoip/geoip2/geolite2/)

we will loose maxmind ASN feature due to limitation of logstash geo plugin (will only read .mmdb, no more .dat)

t3chn0m4g3 commented 8 years ago

Some quick notes why moving to 17.03

no head plugin - see here
docker container only works with workarounds - see here
you cannot specify index options in config file anymore - see here

With Alpha 4 and betas still to come I currently cannot recommend ELK 5 for T-Pot.

MarcinNaw commented 7 years ago

I guess ELK Stack v5 is still in the making. I just wanted to highlight this nice new future:

X-PACK (https://www.elastic.co/products/x-pack/machine-learning) offers now time series anomaly Detection. This tool will automatically alert on unusual changes in a key performance indicator values.

This might come in very handy for automatic detection of new attacks on the honeypot sensors, not only the usual radiation.

t3chn0m4g3 commented 7 years ago

Indeed, ELK 5.x will be baked into 17.06 without X-Pack. Due the licensing scheme it is not suited for an open source project like T-Pot.

funtimes-ninja commented 7 years ago

It would be nice if a key was supplied, that it would auto install it.

t3chn0m4g3 commented 7 years ago

As mentioned earlier, X-Pack will not be part of T-Pot.

melazzouzi commented 7 years ago

Any update when tpot 17.06 will be released.

t3chn0m4g3 commented 7 years ago

@melazzouzi I was planning with a first alpha at the end of June. Some delays in the logstash geo_ip filter (ASN functionality) as well as some new features (vs. time available) might point to a GA release in September or even October, probably as a T-Pot 17.10 😇