logstash.conf changes aren't affecting logs in a distributed install

[I-Simon-I]

Hi !

I've been using T-Pot standalone on multiple VM for a few weeks and I managed to set it up the way I wanted. However I was asked to use the distributed deployement rather than multiple standalone. I installed T-Pot on every VM and linked them all to the HIVE using the deploy.sh command. Everything works fine, logs are coming through but I can't find a way to make the email alerts work on this install. For some reason, my logstash.conf file aren't "used" by logstash. It seems like the logstash in the distributed version is using another logstash.conf other than the two I've modified. Email alerts aren't generated and the tag "_geoip_lookup_failure" is present in every log even though I've disabled it.

That would be very helpful if someone could help me understand where I have to modify the logstash.conf file in order for the logs to be processed (changes described below) the way I need them to be. This would also allow me to better understand how logs are processed between the sensors and the hive.

PS : It's worth noting that whenever I restart tpot on the HIVE, every container goes down and keeps crashing for a few minutes then they all go up. However, when I reboot, it starts normally

After reboot

After systemctl restart tpot

Thx for your time !

- What version of the OS are you currently using lsb_release -a and uname -a?

- What T-Pot version are you currently using?

T-Pot 22.04.0

- What edition (Standard, Nextgen, etc.) of T-Pot are you running?

Distributed

- What architecture are you running on (i.e. hardware, cloud, VM, etc.)?

VM

- Did you have any problems during the install? If yes, please attach /install.log /install.err.

No

- How long has your installation been running?

A few hours

- Did you install upgrades, packages or use the update script?

Yes, on the HIVE SENSOR and the HIVE

- Did you modify any scripts or configs? If yes, please attach the changes.

I've modified the /opt/tpot/docker/elk/logstash/dist/logstash.conf file and made a copy of it under /data/elk on both the HIVE and the HIVE SENSOR and I added the volume in the "Logstash" section of the /opt/tpot/etc/tpot.yml file

Changes :

• Added throttle that adds the tag "email" for the first 10 logs of each honeypot

• Dropped NGINX logs because it was too noisy

• Disabled GEOIP coordinates tags

• Added output settings to send email alerts

• I have disabled the following services :

• Suricata

• Fatt

• P0f

• Spiderfoot

• Added the volume to the container in tpot.yml

- Please provide a screenshot of glances and htop.

The HIVE SENSOR

The HIVE

- How much free disk space is available (df -h)?-

35GB on the HIVE 35GB on the SENSOR

- What is the current container status (dps.sh)?

The HIVE SENSOR

The HIVE

- What is the status of the T-Pot service (systemctl status tpot)?

The HIVE SENSOR

The HIVE

- What ports are being occupied? Stop T-Pot systemctl stop tpot and run netstat -tulpen

The HIVE SENSOR

The HIVE

[t3chn0m4g3]

Please follow this comment.

[I-Simon-I]

I don't want to export the logs to another endpoint, I wanna keep it within the main HIVE but I would like to apply filters and use logstash output to generate alerts but the logstash.conf file doesn't seem to work as it did for the standalone version