search

mailcow / mailcow-dockerized

mailcow: dockerized - 🐮 + 🐋 = 💕
https://mailcow.email
GNU General Public License v3.0
8.71k stars 1.17k forks source link

Fail2ban not unbanning IP addresses #5879

Open mrclschstr opened 4 months ago

mrclschstr commented 4 months ago

Contribution guidelines

I've found a bug and checked that ...

Description

Currently I get a lot of login requests to my mailcow instance from a certain subnet, which is why the netfilter container does a lot of bans and unbans. I have noticed that after a certain time (about 3 to 5 days) netfilter no longer unbans the IPs and they remain permanently banned, so to speak. Here is an example for the IP 194.169.175.10:

netfilter-mailcow-1  | Banning 194.169.175.10/32 for 166 minutes
netfilter-mailcow-1  | Unbanning 194.169.175.10/32
netfilter-mailcow-1  | Banning 194.169.175.10/32 for 166 minutes
netfilter-mailcow-1  | Unbanning 194.169.175.10/32
netfilter-mailcow-1  | Banning 194.169.175.10/32 for 166 minutes
netfilter-mailcow-1  | Unbanning 194.169.175.10/32
netfilter-mailcow-1  | Banning 194.169.175.10/32 for 166 minutes
netfilter-mailcow-1  | Unbanning 194.169.175.10/32
netfilter-mailcow-1  | Banning 194.169.175.10/32 for 166 minutes

In the mailcow GUI, the IPs are displayed with a negative ban time:

grafik

I have not changed anything in the netfilter settings, so they are default. Restarting the netfilter container usually helps to unblock the IP addresses again.

The following issue describes similar symptoms: https://github.com/mailcow/mailcow-dockerized/issues/5518

Logs:

See description above.

Steps to reproduce:

I don't know how to reproduce the error specifically.

Which branch are you using?

master

Which architecture are you using?

x86

Operating System:

Ubuntu 22.04.4 LTS

Server/VM specifications:

4 CPU, 8 GB RAM

Is Apparmor, SELinux or similar active?

No

Virtualization technology:

KVM

Docker version:

26.1.1

docker-compose version or docker compose version:

v2.27.0

mailcow version:

2024-04

Reverse proxy:

No

Logs of git diff:

diff --git a/data/assets/ssl-example/cert.pem b/data/assets/ssl-example/cert.pem
index 96d16bec..0ff09842 100644
--- a/data/assets/ssl-example/cert.pem
+++ b/data/assets/ssl-example/cert.pem
@@ -1,19 +1,33 @@
 -----BEGIN CERTIFICATE-----
XXX
 -----END CERTIFICATE-----
diff --git a/data/assets/ssl-example/key.pem b/data/assets/ssl-example/key.pem
index cedf35a0..e94ac2d0 100644
--- a/data/assets/ssl-example/key.pem
+++ b/data/assets/ssl-example/key.pem
@@ -1,27 +1,52 @@
------BEGIN RSA PRIVATE KEY-----
XXX
------END RSA PRIVATE KEY-----
+-----BEGIN PRIVATE KEY-----
XXX
+-----END PRIVATE KEY-----
diff --git a/data/conf/postfix/main.cf b/data/conf/postfix/main.cf
index 572300db..2158fecc 100644
--- a/data/conf/postfix/main.cf
+++ b/data/conf/postfix/main.cf
@@ -173,3 +173,36 @@ parent_domain_matches_subdomains = debug_peer_list,fast_flush_domains,mynetworks

 # DO NOT EDIT ANYTHING BELOW #
 # Overrides #
+
+postscreen_dnsbl_sites = wl.mailspike.net=127.0.0.[18;19;20]*-2
+  hostkarma.junkemailfilter.com=127.0.0.1*-2
+  list.dnswl.org=127.0.[0..255].0*-2
+  list.dnswl.org=127.0.[0..255].1*-4
+  list.dnswl.org=127.0.[0..255].2*-6
+  list.dnswl.org=127.0.[0..255].3*-8
+  ix.dnsbl.manitu.net*2
+  bl.spamcop.net*2
+  bl.suomispam.net*2
+  hostkarma.junkemailfilter.com=127.0.0.2*3
+  hostkarma.junkemailfilter.com=127.0.0.4*2
+  hostkarma.junkemailfilter.com=127.0.1.2*1
+  backscatter.spameatingmonkey.net*2
+  bl.ipv6.spameatingmonkey.net*2
+  bl.spameatingmonkey.net*2
+  b.barracudacentral.org=127.0.0.2*7
+  bl.mailspike.net=127.0.0.2*5
+  bl.mailspike.net=127.0.0.[10;11;12]*4
+  dnsbl.sorbs.net=127.0.0.10*8
+  dnsbl.sorbs.net=127.0.0.5*6
+  dnsbl.sorbs.net=127.0.0.7*3
+  dnsbl.sorbs.net=127.0.0.8*2
+  dnsbl.sorbs.net=127.0.0.6*2
+  dnsbl.sorbs.net=127.0.0.9*2
+  zen.spamhaus.org=127.0.0.[10;11]*8
+  zen.spamhaus.org=127.0.0.[4..7]*6
+  zen.spamhaus.org=127.0.0.3*4
+  zen.spamhaus.org=127.0.0.2*3
+
+# User Overrides
+myhostname = mail.XXX.com

Logs of iptables -L -vn:

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
2352K  873M MAILCOW    all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
  10M 3400M MAILCOW    all  --  *      *       0.0.0.0/0            0.0.0.0/0
  26M 8367M DOCKER-USER  all  --  *      *       0.0.0.0/0            0.0.0.0/0
  23M 7145M DOCKER-ISOLATION-STAGE-1  all  --  *      *       0.0.0.0/0            0.0.0.0/0
  18M 6371M ACCEPT     all  --  *      br-mailcow  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
1880K  120M DOCKER     all  --  *      br-mailcow  0.0.0.0/0            0.0.0.0/0
3059K  653M ACCEPT     all  --  br-mailcow !br-mailcow  0.0.0.0/0            0.0.0.0/0
1880K  120M ACCEPT     all  --  br-mailcow br-mailcow  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  docker0 docker0  0.0.0.0/0            0.0.0.0/0

Chain OUTPUT (policy ACCEPT 364K packets, 38M bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.249         tcp dpt:6379
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.5           tcp dpt:8983
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.6           tcp dpt:3306
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.8           tcp dpt:443
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.8           tcp dpt:80
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.250         tcp dpt:12345
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.250         tcp dpt:4190
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.250         tcp dpt:995
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.250         tcp dpt:993
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.250         tcp dpt:143
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.250         tcp dpt:110
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.253         tcp dpt:587
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.253         tcp dpt:465
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.253         tcp dpt:25

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
 pkts bytes target     prot opt in     out     source               destination
3059K  653M DOCKER-ISOLATION-STAGE-2  all  --  br-mailcow !br-mailcow  0.0.0.0/0            0.0.0.0/0
    0     0 DOCKER-ISOLATION-STAGE-2  all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0
  23M 7145M RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain DOCKER-ISOLATION-STAGE-2 (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DROP       all  --  *      br-mailcow  0.0.0.0/0            0.0.0.0/0
    0     0 DROP       all  --  *      docker0  0.0.0.0/0            0.0.0.0/0
3059K  653M RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain DOCKER-USER (1 references)
 pkts bytes target     prot opt in     out     source               destination
  23M 7145M RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain MAILCOW (2 references)
 pkts bytes target     prot opt in     out     source               destination
  370 22144 REJECT     all  --  *      *       194.169.175.20       0.0.0.0/0            reject-with icmp-port-unreachable
 3671  220K REJECT     all  --  *      *       194.169.175.10       0.0.0.0/0            reject-with icmp-port-unreachable
 5832  351K REJECT     all  --  *      *       194.169.175.17       0.0.0.0/0            reject-with icmp-port-unreachable
    0     0 DROP       tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            0.0.0.0/0            multiport dports 3306,6379,8983,12345

Logs of ip6tables -L -vn:

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
 668K  149M MAILCOW    all      *      *       ::/0                 ::/0

Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
3522K 2338M MAILCOW    all      *      *       ::/0                 ::/0
9878K 5946M DOCKER-USER  all      *      *       ::/0                 ::/0
7035K 5587M DOCKER-ISOLATION-STAGE-1  all      *      *       ::/0                 ::/0
4065K 4863M DOCKER     all      *      br-mailcow  ::/0                 ::/0
3412K 4817M ACCEPT     all      *      br-mailcow  ::/0                 ::/0                 ctstate RELATED,ESTABLISHED
2970K  724M ACCEPT     all      br-mailcow !br-mailcow  ::/0                 ::/0
 653K   46M ACCEPT     all      br-mailcow br-mailcow  ::/0                 ::/0

Chain OUTPUT (policy ACCEPT 49739 packets, 452M bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain DOCKER (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::c  tcp dpt:443
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::c  tcp dpt:80
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::b  tcp dpt:995
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::b  tcp dpt:110
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::b  tcp dpt:143
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::b  tcp dpt:4190
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::b  tcp dpt:993
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::10  tcp dpt:25
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::10  tcp dpt:465
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::10  tcp dpt:587

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
 pkts bytes target     prot opt in     out     source               destination
2970K  724M DOCKER-ISOLATION-STAGE-2  all      br-mailcow !br-mailcow  ::/0                 ::/0
7035K 5587M RETURN     all      *      *       ::/0                 ::/0

Chain DOCKER-ISOLATION-STAGE-2 (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DROP       all      *      br-mailcow  ::/0                 ::/0
2970K  724M RETURN     all      *      *       ::/0                 ::/0

Chain DOCKER-USER (1 references)
 pkts bytes target     prot opt in     out     source               destination
7035K 5587M RETURN     all      *      *       ::/0                 ::/0

Chain MAILCOW (2 references)
 pkts bytes target     prot opt in     out     source               destination

Logs of iptables -L -vn -t nat:

Chain PREROUTING (policy ACCEPT 730K packets, 133M bytes)
 pkts bytes target     prot opt in     out     source               destination
 386K   20M DOCKER     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ADDRTYPE match dst-type LOCAL

Chain INPUT (policy ACCEPT 12068 packets, 695K bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 481 packets, 38170 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DOCKER     all  --  *      *       0.0.0.0/0           !127.0.0.0/8          ADDRTYPE match dst-type LOCAL

Chain POSTROUTING (policy ACCEPT 223K packets, 14M bytes)
 pkts bytes target     prot opt in     out     source               destination
1366K  104M MASQUERADE  all  --  *      !br-mailcow  172.22.1.0/24        0.0.0.0/0
    0     0 MASQUERADE  all  --  *      !docker0  172.17.0.0/16        0.0.0.0/0
    0     0 MASQUERADE  tcp  --  *      *       172.22.1.249         172.22.1.249         tcp dpt:6379
    0     0 MASQUERADE  tcp  --  *      *       172.22.1.5           172.22.1.5           tcp dpt:8983
    0     0 MASQUERADE  tcp  --  *      *       172.22.1.6           172.22.1.6           tcp dpt:3306
    0     0 MASQUERADE  tcp  --  *      *       172.22.1.8           172.22.1.8           tcp dpt:443
    0     0 MASQUERADE  tcp  --  *      *       172.22.1.8           172.22.1.8           tcp dpt:80
    0     0 MASQUERADE  tcp  --  *      *       172.22.1.250         172.22.1.250         tcp dpt:12345
    0     0 MASQUERADE  tcp  --  *      *       172.22.1.250         172.22.1.250         tcp dpt:4190
    0     0 MASQUERADE  tcp  --  *      *       172.22.1.250         172.22.1.250         tcp dpt:995
    0     0 MASQUERADE  tcp  --  *      *       172.22.1.250         172.22.1.250         tcp dpt:993
    0     0 MASQUERADE  tcp  --  *      *       172.22.1.250         172.22.1.250         tcp dpt:143
    0     0 MASQUERADE  tcp  --  *      *       172.22.1.250         172.22.1.250         tcp dpt:110
    0     0 MASQUERADE  tcp  --  *      *       172.22.1.253         172.22.1.253         tcp dpt:587
    0     0 MASQUERADE  tcp  --  *      *       172.22.1.253         172.22.1.253         tcp dpt:465
    0     0 MASQUERADE  tcp  --  *      *       172.22.1.253         172.22.1.253         tcp dpt:25

Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination
    2   120 RETURN     all  --  br-mailcow *       0.0.0.0/0            0.0.0.0/0
    0     0 RETURN     all  --  docker0 *       0.0.0.0/0            0.0.0.0/0
    0     0 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            127.0.0.1            tcp dpt:7654 to:172.22.1.249:6379
    0     0 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            127.0.0.1            tcp dpt:18983 to:172.22.1.5:8983
    0     0 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            127.0.0.1            tcp dpt:13306 to:172.22.1.6:3306
20164 1101K DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:443 to:172.22.1.8:443
 9306  439K DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:80 to:172.22.1.8:80
    0     0 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            127.0.0.1            tcp dpt:19991 to:172.22.1.250:12345
   19  1048 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:4190 to:172.22.1.250:4190
  382 22800 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:995 to:172.22.1.250:995
 1069 64080 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:993 to:172.22.1.250:993
 1202 71968 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:143 to:172.22.1.250:143
   77  4300 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:110 to:172.22.1.250:110
  112  6128 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:587 to:172.22.1.253:587
 7032  422K DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:465 to:172.22.1.253:465
  865 50028 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:25 to:172.22.1.253:25

Logs of ip6tables -L -vn -t nat:

Chain PREROUTING (policy ACCEPT 67598 packets, 6046K bytes)
 pkts bytes target     prot opt in     out     source               destination
88988 7396K DOCKER     all      *      *       ::/0                 ::/0                 ADDRTYPE match dst-type LOCAL

Chain INPUT (policy ACCEPT 39 packets, 2970 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 516 packets, 47824 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DOCKER     all      *      *       ::/0                !::1                  ADDRTYPE match dst-type LOCAL

Chain POSTROUTING (policy ACCEPT 26944 packets, 2164K bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 MASQUERADE  all      *      br-mailcow  ::/0                 ::/0                 ADDRTYPE match dst-type LOCAL
 366K   35M MASQUERADE  all      *      !br-mailcow  fd4d:6169:6c63:6f77::/64  ::/0
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::c  fd4d:6169:6c63:6f77::c  tcp dpt:443
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::c  fd4d:6169:6c63:6f77::c  tcp dpt:80
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::b  fd4d:6169:6c63:6f77::b  tcp dpt:995
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::b  fd4d:6169:6c63:6f77::b  tcp dpt:110
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::b  fd4d:6169:6c63:6f77::b  tcp dpt:143
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::b  fd4d:6169:6c63:6f77::b  tcp dpt:4190
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::b  fd4d:6169:6c63:6f77::b  tcp dpt:993
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::10  fd4d:6169:6c63:6f77::10  tcp dpt:25
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::10  fd4d:6169:6c63:6f77::10  tcp dpt:465
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::10  fd4d:6169:6c63:6f77::10  tcp dpt:587

Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination
  288 23040 RETURN     all      br-mailcow *       ::/0                 ::/0
 1320  102K DNAT       tcp      !br-mailcow *       ::/0                 ::/0                 tcp dpt:443 to:[fd4d:6169:6c63:6f77::c]:443
  329 25504 DNAT       tcp      !br-mailcow *       ::/0                 ::/0                 tcp dpt:80 to:[fd4d:6169:6c63:6f77::c]:80
   11   880 DNAT       tcp      !br-mailcow *       ::/0                 ::/0                 tcp dpt:995 to:[fd4d:6169:6c63:6f77::b]:995
    0     0 DNAT       tcp      !br-mailcow *       ::/0                 ::/0                 tcp dpt:110 to:[fd4d:6169:6c63:6f77::b]:110
    0     0 DNAT       tcp      !br-mailcow *       ::/0                 ::/0                 tcp dpt:143 to:[fd4d:6169:6c63:6f77::b]:143
    1    80 DNAT       tcp      !br-mailcow *       ::/0                 ::/0                 tcp dpt:4190 to:[fd4d:6169:6c63:6f77::b]:4190
  989 81453 DNAT       tcp      !br-mailcow *       ::/0                 ::/0                 tcp dpt:993 to:[fd4d:6169:6c63:6f77::b]:993
    6   420 DNAT       tcp      !br-mailcow *       ::/0                 ::/0                 tcp dpt:25 to:[fd4d:6169:6c63:6f77::10]:25
    2   144 DNAT       tcp      !br-mailcow *       ::/0                 ::/0                 tcp dpt:465 to:[fd4d:6169:6c63:6f77::10]:465
    1    64 DNAT       tcp      !br-mailcow *       ::/0                 ::/0                 tcp dpt:587 to:[fd4d:6169:6c63:6f77::10]:587

DNS check:

172.64.155.249
104.18.32.7
factor4 commented 3 months ago

I have the same symptoms:

grafik

And the same IP blocked... ;)

itkfm commented 3 months ago

Been suffering from this for months. At least since May 9 2023; that’s when I noticed it and opened Ticket#1253407.

mrclschstr commented 3 months ago

At least since May 9 2023; that’s when I noticed it and opened Ticket#1253407.

I cannot find your ticket number. Are you sure it's correct?

itkfm commented 3 months ago

I cannot find your ticket number. Are you sure it's correct?

I assume, you’re a TINC employee/contractor with access to the servercow.de ticketing system?

Those are my two reports of the issue:

Ticket Number Priority Department Summary
1253407 Medium mailcow Premium Negative Bann-Zeiten und unsichtbare Buttons
1910644 High mailcow Premium Wieder negative Bannzeiten
kovacs-andras commented 3 months ago

If you get a lot of login requests with malicious attempt I would use sg. else than that builtin script. It's really not efficient. Do you have any firewall solutions before your server? Or are you familiar with the "real" fail2ban project? https://github.com/fail2ban/fail2ban

mrclschstr commented 3 months ago

Yes, I'm familiar with the real fail2ban project, but I guess having two systems is one too many? Is there a tutorial on how to install an external fail2ban solution? I didn't find anything in the docs...

By the way: Why does mailcow use a self-scripted fail2ban solution at all?

mkuron commented 3 months ago

Why does mailcow use a self-scripted fail2ban solution at all?

When I wrote the initial version of this script (it has changed a lot since then) in 2017, the "real" fail2ban did not support IPv6 and couldn't read logs from Docker. I'm pretty sure it supports IPv6 nowadays, but I'm not sure it can handle any of Docker's log drivers other than the (non-default) systemd log.

evilMouse commented 3 months ago

I have the same issue, although I've changed default fail2ban parameters strongly: Ban time (s): 86400 (24h) Max. ban time (s): 604800 (one week) Ban time is incremented with each ban: true Max. attempts: 5 Retry window (s) for max. attempts: 86400

Example lines (IPs changed): 152.4.204.217/32 (-493h -35m -40s) - [unban] [whitelist] [blacklist (needs restart)] 81.14.65.107/32 (-495h -11m -19s) - [unban] [whitelist] [blacklist (needs restart)] 94.18.180.171/32 (-465h -43m -10s) - [unban] [whitelist] [blacklist (needs restart)]

I've left it running for a few weeks without restarting and eventually it hanged with the last line visible: Error reading log line from pubsub: 'max_attempts'

mikeyjoel commented 3 months ago

Running into a similar issue. I've used a third party VPN and tried to force brute for testing and netfilter is not logging in the fail attempts making it harder to create alerts on a SIEM and use SOAR to automate my firewall to have the IP/Subnet blocked.

@kovacs-andras We need to have logging with attempt fails to begin with if we want to be able to use a third-party that can facilitate this service if fail2ban does not work. How are you checking logs and automating today for failed login attempts? Do you recommend a specific log source to ingest login events that we can use to not solely rely on fail2ban/netfilter? Thanks in advance!

I started noticing right after the latest release (2024-6a), so I can't tell when this has been occurring.

The entire machine has been restarted and re-tested with the same issue. Everything else is working as expected.

The only difference in my environment is that IPv6 was completely removed following the official doc.

# docker compose logs -f --tail=200 netfilter-mailcow
netfilter-mailcow-1  | # Warning: table ip nat is managed by iptables-nft, do not touch!
netfilter-mailcow-1  | # Warning: table ip filter is managed by iptables-nft, do not touch!
netfilter-mailcow-1  | # Warning: table ip6 nat is managed by iptables-nft, do not touch!
netfilter-mailcow-1  | # Warning: table ip6 filter is managed by iptables-nft, do not touch!
netfilter-mailcow-1  | Using NFTables backend
netfilter-mailcow-1  | Clearing all bans
netfilter-mailcow-1  | Initializing mailcow netfilter chain
netfilter-mailcow-1  | MAILCOW ip chain created successfully.
netfilter-mailcow-1  | MAILCOW ip6 chain created successfully.
netfilter-mailcow-1  | Setting MAILCOW isolation
netfilter-mailcow-1  | Watching Redis channel F2B_CHANNEL
# iptables -L -n
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         
MAILCOW    0    --  0.0.0.0/0            0.0.0.0/0            /* mailcow */
DOCKER-USER  0    --  0.0.0.0/0            0.0.0.0/0           
DOCKER-ISOLATION-STAGE-1  0    --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     0    --  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
DOCKER     0    --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     0    --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     0    --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     0    --  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
DOCKER     0    --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     0    --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     0    --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     0    --  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
DOCKER     0    --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     0    --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     0    --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     0    --  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
DOCKER     0    --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     0    --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     0    --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     0    --  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
DOCKER     0    --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     0    --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     0    --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     0    --  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
DOCKER     0    --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     0    --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     0    --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     0    --  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
DOCKER     0    --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     0    --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     0    --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     0    --  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
DOCKER     0    --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     0    --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     0    --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     0    --  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
DOCKER     0    --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     0    --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     0    --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     0    --  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
DOCKER     0    --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     0    --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     0    --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     0    --  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
DOCKER     0    --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     0    --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     0    --  0.0.0.0/0            0.0.0.0/0           

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         

Chain DOCKER (11 references)
target     prot opt source               destination         
ACCEPT     6    --  0.0.0.0/0            172.17.0.2           tcp dpt:8834
ACCEPT     6    --  0.0.0.0/0            172.18.0.2           tcp dpt:8080
ACCEPT     6    --  0.0.0.0/0            172.17.0.3           tcp dpt:80
ACCEPT     6    --  0.0.0.0/0            192.168.32.2         tcp dpt:22
ACCEPT     6    --  0.0.0.0/0            172.18.0.2           tcp dpt:50000
ACCEPT     6    --  0.0.0.0/0            192.168.112.3        tcp dpt:80
ACCEPT     6    --  0.0.0.0/0            172.19.0.2           tcp dpt:4200
ACCEPT     6    --  0.0.0.0/0            192.168.32.2         tcp dpt:3000
ACCEPT     6    --  0.0.0.0/0            172.21.0.2           tcp dpt:9443
ACCEPT     6    --  0.0.0.0/0            172.23.0.2           tcp dpt:5432
ACCEPT     6    --  0.0.0.0/0            172.20.0.3           tcp dpt:80
ACCEPT     6    --  0.0.0.0/0            192.168.80.3         tcp dpt:80
ACCEPT     17   --  0.0.0.0/0            172.17.0.4           udp dpt:1900
ACCEPT     6    --  0.0.0.0/0            172.23.0.3           tcp dpt:8080
ACCEPT     6    --  0.0.0.0/0            172.17.0.5           tcp dpt:80
ACCEPT     6    --  0.0.0.0/0            172.19.0.2           tcp dpt:9000
ACCEPT     6    --  0.0.0.0/0            172.17.0.6           tcp dpt:3000
ACCEPT     6    --  0.0.0.0/0            172.19.0.3           tcp dpt:8000
ACCEPT     6    --  0.0.0.0/0            172.19.0.2           tcp dpt:9420
ACCEPT     6    --  0.0.0.0/0            172.19.0.3           tcp dpt:8080
ACCEPT     17   --  0.0.0.0/0            172.17.0.4           udp dpt:7359
ACCEPT     6    --  0.0.0.0/0            172.19.0.3           tcp dpt:8086
ACCEPT     6    --  0.0.0.0/0            172.19.0.3           tcp dpt:8089
ACCEPT     6    --  0.0.0.0/0            172.17.0.4           tcp dpt:8096
ACCEPT     6    --  0.0.0.0/0            172.19.0.3           tcp dpt:9997
ACCEPT     6    --  0.0.0.0/0            172.22.1.249         tcp dpt:6379
ACCEPT     6    --  0.0.0.0/0            172.22.1.5           tcp dpt:8983
ACCEPT     6    --  0.0.0.0/0            172.22.1.6           tcp dpt:3306
ACCEPT     6    --  0.0.0.0/0            172.22.1.250         tcp dpt:110
ACCEPT     6    --  0.0.0.0/0            172.22.1.250         tcp dpt:143
ACCEPT     6    --  0.0.0.0/0            172.22.1.250         tcp dpt:993
ACCEPT     6    --  0.0.0.0/0            172.22.1.250         tcp dpt:995
ACCEPT     6    --  0.0.0.0/0            172.22.1.250         tcp dpt:4190
ACCEPT     6    --  0.0.0.0/0            172.22.1.8           tcp dpt:80
ACCEPT     6    --  0.0.0.0/0            172.22.1.250         tcp dpt:12345
ACCEPT     6    --  0.0.0.0/0            172.22.1.8           tcp dpt:443
ACCEPT     6    --  0.0.0.0/0            172.22.1.253         tcp dpt:25
ACCEPT     6    --  0.0.0.0/0            172.22.1.253         tcp dpt:465
ACCEPT     6    --  0.0.0.0/0            172.22.1.253         tcp dpt:587

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
target     prot opt source               destination         
DOCKER-ISOLATION-STAGE-2  0    --  0.0.0.0/0            0.0.0.0/0           
DOCKER-ISOLATION-STAGE-2  0    --  0.0.0.0/0            0.0.0.0/0           
DOCKER-ISOLATION-STAGE-2  0    --  0.0.0.0/0            0.0.0.0/0           
DOCKER-ISOLATION-STAGE-2  0    --  0.0.0.0/0            0.0.0.0/0           
DOCKER-ISOLATION-STAGE-2  0    --  0.0.0.0/0            0.0.0.0/0           
DOCKER-ISOLATION-STAGE-2  0    --  0.0.0.0/0            0.0.0.0/0           
DOCKER-ISOLATION-STAGE-2  0    --  0.0.0.0/0            0.0.0.0/0           
DOCKER-ISOLATION-STAGE-2  0    --  0.0.0.0/0            0.0.0.0/0           
DOCKER-ISOLATION-STAGE-2  0    --  0.0.0.0/0            0.0.0.0/0           
DOCKER-ISOLATION-STAGE-2  0    --  0.0.0.0/0            0.0.0.0/0           
DOCKER-ISOLATION-STAGE-2  0    --  0.0.0.0/0            0.0.0.0/0           
RETURN     0    --  0.0.0.0/0            0.0.0.0/0           

Chain DOCKER-ISOLATION-STAGE-2 (11 references)
target     prot opt source               destination         
DROP       0    --  0.0.0.0/0            0.0.0.0/0           
DROP       0    --  0.0.0.0/0            0.0.0.0/0           
DROP       0    --  0.0.0.0/0            0.0.0.0/0           
DROP       0    --  0.0.0.0/0            0.0.0.0/0           
DROP       0    --  0.0.0.0/0            0.0.0.0/0           
DROP       0    --  0.0.0.0/0            0.0.0.0/0           
DROP       0    --  0.0.0.0/0            0.0.0.0/0           
DROP       0    --  0.0.0.0/0            0.0.0.0/0           
DROP       0    --  0.0.0.0/0            0.0.0.0/0           
DROP       0    --  0.0.0.0/0            0.0.0.0/0           
DROP       0    --  0.0.0.0/0            0.0.0.0/0           
RETURN     0    --  0.0.0.0/0            0.0.0.0/0           

Chain DOCKER-USER (1 references)
target     prot opt source               destination         
RETURN     0    --  0.0.0.0/0            0.0.0.0/0           

Chain MAILCOW (1 references)
target     prot opt source               destination         
DROP       6    --  0.0.0.0/0            0.0.0.0/0            /* mailcow isolation */
factor4 commented 2 months ago

Hi! 5 days ago I migrated my mailcow VM to a new physical host. Migration was done by offline Backup/Restore. The day before I update to 2024-6a. Can't tell what solved it, but since then IP addresses get unbanned as expected.

On the new host several things changed from my previous setup:

Just want to share this. Maybe it help somebody...

itkfm commented 2 months ago

Can't tell what solved it, but since then IP addresses get unbanned as expected.

Watch out for whether this occurs only occasionally after a while – like it does for us.

factor4 commented 2 months ago

Watch out for whether this occurs only occasionally after a while – like it does for us.

You're right. It was too early to celebrate. The phenomenon still exists for me too...

PhilippeAccorsi commented 1 month ago

We have same kind of symptom. When click on unban button, the frontend show unban pending but IP never unban.

I need to execute command like this docker exec netfilter-mailcow-1 iptables -D MAILCOW -s x.x.x.x/32 -j DROP to solve the problem.

itkfm commented 1 month ago

I need to execute command like this […]

Does restarting the netfilter container not work for you? Does your command actually resolve the problem or just get the specified rule off the list?