Attack Map stops showing events

DRIgnazGortngschirl commented 1 year ago

What version of the OS are you currently using lsb_release -a and uname -a?

[root@redundantblackfish:/home/tsec]# lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 11 (bullseye)
Release:        11
Codename:       bullseye
[root@redundantblackfish:/home/tsec]# uname -a
Linux redundantblackfish 5.10.0-23-amd64 #1 SMP Debian 5.10.179-1 (2023-05-12) x86_64 GNU/Linux

What T-Pot version are you currently using? 22.04.0 ISO based installation
What edition (Standard, Nextgen, etc.) of T-Pot are you running? Standard I guess
What architecture are you running on (i.e. hardware, cloud, VM, etc.)? VM / ISO based
Did you have any problems during the install? If yes, please attach /install.log /install.err. no
How long has your installation been running? Good, but this issue keeps reappearing
Did you install upgrades, packages or use the update script? No
Did you modify any scripts or configs? If yes, please attach the changes. Only the /etc/networking/interfaces from an DHCP to a static one And the /etc/docker deamon.json to overwire the default networks as it overlaps with existing ones in our infrastructure.
Please provide a screenshot of glances and htop.
How much free disk space is available (df -h)? ~164 GB

What is the current container status (dps.sh)?


[root@redundantblackfish:/home/tsec]# dps.sh
[ ========| System |======== ]
 DATE:  Thu 06 Jul 2023 09:21:50 AM CEST
UPTIME:   09:21:50 up  6:59,  2 users,  load average: 3.90, 4.51, 4.45
T-POT:ACTIVE
BLACKHOLE:  DISABLED

NAME STATUS PORTS adbhoney Up 7 hours (healthy) 0.0.0.0:5555->5555/tcp ciscoasa Up 7 hours 0.0.0.0:5000->5000/udp, 0.0.0.0:8443->8443/tcp citrixhoneypot Up 7 hours 0.0.0.0:443->443/tcp conpot_guardian_ast Up 7 hours (healthy) 0.0.0.0:10001->10001/tcp conpot_iec104 Up 7 hours (healthy) 0.0.0.0:161->161/udp, 0.0.0.0:2404->2404/tcp conpot_ipmi Up 7 hours (healthy) 0.0.0.0:623->623/udp conpot_kamstrup_382 Up 7 hours (healthy) 0.0.0.0:1025->1025/tcp, 0.0.0.0:50100->50100/tcp cowrie Up 7 hours 0.0.0.0:22-23->22-23/tcp ddospot Up 7 hours 0.0.0.0:19->19/udp, 0.0.0.0:53->53/udp, 0.0.0.0:123->123/udp, 0.0.0.0:1900->1900/udp dicompot Up 7 hours 0.0.0.0:11112->11112/tcp dionaea Up 7 hours (healthy) 0.0.0.0:20-21->20-21/tcp, 0.0.0.0:42->42/tcp, 0.0.0.0:81->81/tcp, 0.0.0.0:135->135/tcp, 0.0.0.0:445->445/tcp, 0.0.0.0:1433->1433/tcp, 0.0.0.0:1723->1723/tcp, 0.0.0.0:1883->1883/tcp, 0.0.0.0:3306->3306/tcp, 0.0.0.0:27017->27017/tcp, 0.0.0.0:69->69/udp elasticpot Up 7 hours 0.0.0.0:9200->9200/tcp elasticsearch Up 7 hours (healthy) 127.0.0.1:64298->9200/tcp ewsposter Up 7 hours fatt Up 7 hours heralding Up 7 hours 0.0.0.0:110->110/tcp, 0.0.0.0:143->143/tcp, 0.0.0.0:465->465/tcp, 0.0.0.0:993->993/tcp, 0.0.0.0:995->995/tcp, 0.0.0.0:1080->1080/tcp, 0.0.0.0:5432->5432/tcp, 0.0.0.0:5900->5900/tcp honeytrap Up 7 hours ipphoney Up 7 hours 0.0.0.0:631->631/tcp kibana Up 7 hours (healthy) 127.0.0.1:64296->5601/tcp logstash Up 7 hours (healthy) mailoney Up 7 hours 0.0.0.0:25->25/tcp map_data Up 7 hours map_redis Up 7 hours map_web Up 7 hours 127.0.0.1:64299->64299/tcp medpot Up 7 hours 0.0.0.0:2575->2575/tcp nginx Up 7 hours p0f Up 7 hours redishoneypot Up 7 hours 0.0.0.0:6379->6379/tcp sentrypeer Up 7 hours 0.0.0.0:5060->5060/udp snare Up 7 hours 0.0.0.0:80->80/tcp spiderfoot Up 7 hours (healthy) 127.0.0.1:64303->8080/tcp suricata Up 7 hours tanner Up 7 hours tanner_api Up 7 hours tanner_phpox Up 7 hours tanner_redis Up 7 hours

- What is the status of the T-Pot service (`systemctl status tpot`)?
active (running)
- What ports are being occupied? Stop T-Pot `systemctl stop tpot` and run `netstat -tulpen`

[root@redundantblackfish:/home/tsec]#netstat -tulpen Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State User Inode PID/Program name tcp 0 0 127.0.0.1:41297 0.0.0.0: LISTEN 0 2224017 2606522/cockpit-bri tcp 0 0 127.0.0.1:35703 0.0.0.0: LISTEN 0 12818 732/containerd tcp 0 0 127.0.0.1:43749 0.0.0.0: LISTEN 1000 2229689 2606435/cockpit-bri tcp 0 0 0.0.0.0:64295 0.0.0.0: LISTEN 0 12751 744/sshd: /usr/sbin tcp6 0 0 :::64294 ::: LISTEN 0 10314 1/init tcp6 0 0 :::64295 ::: LISTEN 0 12753 744/sshd: /usr/sbin udp 0 0 127.0.0.1:323 0.0.0.0: 0 11575 735/chronyd udp6 0 0 ::1:323 ::: 0 11576 735/chronyd


- If a single container shows as `DOWN` you can run `docker logs <container-name>` for the latest log entries
No

I do face the issue after some days / hours the web interface needs a restart, just now tested a reboot of the whole server / reboot just with systemctl restart tpot and I still want to test if a restart of the nginx container would solve the issue. 

Also tried different browsers from different networks to access the map.

![image](https://github.com/telekom-security/tpotce/assets/30075959/da96f3dc-f935-4d9d-b694-7950c41a1d9a)

DRIgnazGortngschirl commented 1 year ago

I just noticed after having the Attack Map open for some time that after a huge amount of events happened all stopped and now nothing works any more like the described issue, is there some limitation of possible display of events ?

DRIgnazGortngschirl commented 1 year ago

Restarting map_web container helps for sometime until it stops again.

t3chn0m4g3 commented 1 year ago

I cannot reproduce this on Chrome, Edge and Brave. I know that Safari is slower in handling volumes of traffic and Firefox has issues with Leaflet. Based on your screenshots, both browsers will not connect since either the AttackMapServer or the DataServer for the AttackMap is seemingly not started or cannot collect events from ES (the counters on T-Pot Stats are not initialized and should show at least "0" if connection is ok).

t3chn0m4g3 commented 1 year ago

I just pushed a new docker image for AttackMap 2.1.0 which handles errors more gracefully including green / red indicator in the upper left corner.

DRIgnazGortngschirl commented 1 year ago

I just pushed a new docker image for AttackMap 2.1.0 which handles errors more gracefully including green / red indicator in the upper left corner.

Thanks, @t3chn0m4g3 I will check it out tomorrow and let you know how it goes!

DRIgnazGortngschirl commented 1 year ago

@t3chn0m4g3 Do I even need to do anything as the images are pulled every day with the cron job or ?

t3chn0m4g3 commented 1 year ago

If you see a green / red label T-Pot Honeypot Stats then the images have been pulled just fine.

telekom-security / tpotce

Attack Map stops showing events #1374