Hello fellow HELK users, I could use some syslog/firewall help with HELK.
I am running the HELK, with winlogbeats with the Sysmon dumping to Kafka flawlessly.
My problem is with my Forti firewall, and it having to use Syslog.
I am able to receive the syslogs, and ingest them to an index. The data is ugly as can be to the logs-indexme*, which is another thing I wouldn't want it to be.
I have tried finding other solutions on the internet, but every time I find them, I just break it all and have to end up reverting after several lost hours. This has occurred multiple times.
The fixes I have tried has been changing input.conf and output.conf and also added in filters to filter files, which I am lost if I am putting them in the correct place, it just breaks everything, and decided to better leave it the way it is for now with it at least ingesting.
Get operating system and version
for linux (except Mac) use:
NAME="Ubuntu"
VERSION="20.04.2 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.2 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
Get disk space, memory, processor cores, and docker storage
Docker Space:
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 196G 39G 147G 21% /
Memory:
total used free shared buff/cache available
Mem: 15 11 0 0 4 3
Swap: 3 0 3
Cores:
4
Get output of the HELK docker containers:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS
NAMES
2092979b3cd9 confluentinc/ksqldb-cli:latest "/bin/sh" 8 days ago Up About an hour helk-ksql-cli
776b063d4784 confluentinc/ksqldb-server:latest "/usr/bin/docker/run" 8 days ago Up About an hour 0.0.0.0:8088->8088/tcp, :::8088->8088/tcp helk-ksql-server
75f273f78de0 otrf/helk-kafka-broker:2.4.0 "./kafka-entrypoint.…" 8 days ago Up About an hour 0.0.0.0:9092->9092/tcp, :::9092->9092/tcp helk-kafka-broker
7b315bb40dbb otrf/helk-spark-worker:2.4.5 "./spark-worker-entr…" 8 days ago Up About an hour helk-spark-worker
076850dc6b55 otrf/helk-zookeeper:2.4.0 "./zookeeper-entrypo…" 8 days ago Up About an hour 2181/tcp, 2888/tcp, 3888/tcp helk-zookeeper
1aff5c7bdb3e otrf/helk-spark-master:2.4.5 "./spark-master-entr…" 8 days ago Up About an hour 7077/tcp, 0.0.0.0:8080->8080/tcp, :::8080->8080/tcp
helk-spark-master
61028c083109 docker_helk-jupyter "/opt/jupyter/script…" 8 days ago Up About an hour 8000/tcp, 8888/tcp helk-jupyter
e2b2d0f94e9b otrf/helk-elastalert:latest "./elastalert-entryp…" 8 days ago Up About an hour helk-elastalert
6959eed9c5ef otrf/helk-nginx:0.3.0 "/opt/helk/scripts/n…" 8 days ago Up About an hour 0.0.0.0:80->80/tcp, :::80->80/tcp, 0.0.0.0:443->443/tcp, :::443->443/tcp helk-nginx
469c5ebf128d otrf/helk-logstash:7.6.2.1 "/usr/share/logstash…" 8 days ago Up About an hour 0.0.0.0:3515->3515/tcp, :::3515->3515/tcp, 0.0.0.0:5044->5044/tcp, :::5044->5044/tcp, 0.0.0.0:5514->5514/tcp, 0.0.0.0:5514->5514/udp, :::5514->5514/tcp, :::5514->5514/udp, 0.0.0.0:8515-8516->8515-8516/tcp, :::8515-8516->8515-8516/tcp, 0.0.0.0:8531->8531/tcp, :::8531->8531/tcp, 0.0.0.0:8515-8516->8515-8516/udp, :::8515-8516->8515-8516/udp, 9600/tcp helk-logstash
00d82738009d docker.elastic.co/kibana/kibana:7.6.2 "/usr/share/kibana/s…" 8 days ago Up About an hour 5601/tcp helk-kibana
44a40e962088 docker.elastic.co/elasticsearch/elasticsearch:7.6.2 "/usr/share/elastics…" 8 days ago Up About an hour 9200/tcp, 9300/tcp helk-elasticsearch
What version of HELK are you using
run the command from within the HELK repo run git log -1 --oneline
Describe the problem
Hello fellow HELK users, I could use some syslog/firewall help with HELK.
I am running the HELK, with winlogbeats with the Sysmon dumping to Kafka flawlessly.
My problem is with my Forti firewall, and it having to use Syslog.
I am able to receive the syslogs, and ingest them to an index. The data is ugly as can be to the logs-indexme*, which is another thing I wouldn't want it to be.
I have tried finding other solutions on the internet, but every time I find them, I just break it all and have to end up reverting after several lost hours. This has occurred multiple times.
The fixes I have tried has been changing input.conf and output.conf and also added in filters to filter files, which I am lost if I am putting them in the correct place, it just breaks everything, and decided to better leave it the way it is for now with it at least ingesting.
What version of HELK are you using
run the command from within the HELK repo run
git log -1 --onelineAny additional context or input you have
pictures, comments, etc.