docker-mailserver / docker-mailserver

Production-ready fullstack but simple mail server (SMTP, IMAP, LDAP, Antispam, Antivirus, etc.) running inside a container.
https://docker-mailserver.github.io/docker-mailserver/latest/
MIT License
14.21k stars 1.79k forks source link

Can't block Spambots #1810

Closed BlockI0tChain closed 3 years ago

BlockI0tChain commented 3 years ago

Subject

I'd like some feedback regarding the Postscreen/Fail2ban configuration.

Description

Hi guys please help me understand what is wrongly configured here with the Postscreen and Fail2ban modules. I've attached a log sample. Fail2ban is activated, it just doesn't kick in to ban the IP's retrying to connect. I can manually ban these IPs with fail2ban-client so the jails are working, however the ban doesn't occur automatically after the "maxretry" is reached. So what is going on here?

# CODE GOES HERE
mailserver       | Feb 15 19:31:21 mail postfix/postscreen[10972]: CONNECT from [87.246.7.226]:40982 to [192.168.176.2]:25
mailserver       | Feb 15 19:31:21 mail postfix/postscreen[10972]: cache btree:/var/lib/postfix/postscreen_cache full cleanup: retained=26 dropped=3 entries
mailserver       | Feb 15 19:31:22 mail postfix/dnsblog[10976]: addr 87.246.7.226 listed by domain list.dnswl.org as 127.0.10.3
mailserver       | Feb 15 19:31:22 mail postfix/dnsblog[10975]: addr 87.246.7.226 listed by domain bl.mailspike.net as 127.0.0.2
mailserver       | Feb 15 19:31:22 mail postfix/postscreen[10972]: PASS NEW [87.246.7.226]:40982
mailserver       | Feb 15 19:31:22 mail postfix/smtpd[10984]: warning: hostname net6-ip226.linkbg.com does not resolve to address 87.246.7.226: Name or service not known
mailserver       | Feb 15 19:31:22 mail postfix/smtpd[10984]: connect from unknown[87.246.7.226]
mailserver       | Feb 15 19:31:24 mail dovecot: auth: passwd-file(ns2@blkchaintech.de,87.246.7.226): unknown user (SHA1 of given password: e15e2f)
mailserver       | Feb 15 19:31:26 mail postfix/smtpd[10984]: warning: unknown[87.246.7.226]: SASL LOGIN authentication failed: UGFzc3dvcmQ6
mailserver       | Feb 15 19:31:26 mail postfix/smtpd[10984]: disconnect from unknown[87.246.7.226] ehlo=1 auth=0/1 rset=1 quit=1 commands=3/4
mailserver       | Feb 15 19:32:07 mail postfix/postscreen[10972]: CONNECT from [87.246.7.226]:49306 to [192.168.176.2]:25
mailserver       | Feb 15 19:32:07 mail postfix/postscreen[10972]: PASS OLD [87.246.7.226]:49306
mailserver       | Feb 15 19:32:07 mail postfix/smtpd[10984]: warning: hostname net6-ip226.linkbg.com does not resolve to address 87.246.7.226: Name or service not known
mailserver       | Feb 15 19:32:07 mail postfix/smtpd[10984]: connect from unknown[87.246.7.226]
mailserver       | Feb 15 19:32:11 mail dovecot: auth: passwd-file(darwin@blkchaintech.de,87.246.7.226): unknown user (SHA1 of given password: 0c0655)
mailserver       | Feb 15 19:32:13 mail postfix/smtpd[10984]: warning: unknown[87.246.7.226]: SASL LOGIN authentication failed: UGFzc3dvcmQ6
mailserver       | Feb 15 19:32:13 mail postfix/smtpd[10984]: disconnect from unknown[87.246.7.226] ehlo=1 auth=0/1 rset=1 quit=1 commands=3/4
mailserver       | Feb 15 19:32:54 mail postfix/postscreen[10972]: CONNECT from [87.246.7.226]:57546 to [192.168.176.2]:25
mailserver       | Feb 15 19:32:54 mail postfix/dnsblog[10977]: addr 87.246.7.226 listed by domain bl.mailspike.net as 127.0.0.2
mailserver       | Feb 15 19:32:54 mail postfix/dnsblog[10974]: addr 87.246.7.226 listed by domain list.dnswl.org as 127.0.10.3
mailserver       | Feb 15 19:32:54 mail postfix/postscreen[10972]: PASS OLD [87.246.7.226]:57546
mailserver       | Feb 15 19:32:54 mail postfix/smtpd[10984]: warning: hostname net6-ip226.linkbg.com does not resolve to address 87.246.7.226: Name or service not known
mailserver       | Feb 15 19:32:54 mail postfix/smtpd[10984]: connect from unknown[87.246.7.226]
mailserver       | Feb 15 19:32:59 mail dovecot: auth: passwd-file(net@blkchaintech,87.246.7.226): unknown user (SHA1 of given password: 976950)
mailserver       | Feb 15 19:33:01 mail postfix/smtpd[10984]: warning: unknown[87.246.7.226]: SASL LOGIN authentication failed: UGFzc3dvcmQ6
mailserver       | Feb 15 19:38:01 mail postfix/smtpd[10984]: timeout after AUTH from unknown[87.246.7.226]
mailserver       | Feb 15 19:38:01 mail postfix/smtpd[10984]: disconnect from unknown[87.246.7.226] ehlo=1 auth=0/1 rset=1 commands=2/3
mailserver       | Feb 15 19:41:21 mail postfix/anvil[10986]: statistics: max connection rate 2/60s for (smtpd:87.246.7.226) at Feb 15 18:32:07
mailserver       | Feb 15 19:41:21 mail postfix/anvil[10986]: statistics: max connection count 1 for (smtpd:87.246.7.226) at Feb 15 18:31:22
mailserver       | Feb 15 19:41:21 mail postfix/anvil[10986]: statistics: max cache size 1 at Feb 15 18:31:22
[87.246.7.226
georglauterbach commented 3 years ago

What does you compose file look like? Maybe a

cap_add: [ "NET_ADMIN" ]

missing?

BlockI0tChain commented 3 years ago

The params: " cap_add:

georglauterbach commented 3 years ago

cc @casperklein

Is F2B even configured to do this by default?

BlockI0tChain commented 3 years ago

fail2ban-jail.cf is added with the default config.

georglauterbach commented 3 years ago

I'm not a F2B expert, maybe @casperklein knows more.

BlockI0tChain commented 3 years ago

Got it.
Could the Postscreen be the issue here though? Just curious about this line with the whitelist cache getting full:

mailserver       | Feb 15 19:31:21 mail postfix/postscreen[10972]: cache btree:/var/lib/postfix/postscreen_cache full cleanup: retained=26 dropped=3 entries

Also: Would this work here as a quickfix? *enable a Whitelist/Blacklist in "/etc/postfix/main.cf"

      postscreen_access_list = permit_mynetworks cidr:/etc/postfix/postscreen_access.cidr
      postscreen_blacklist_action = drop

*Add/Reject cidr in "/etc/postfix/postscreen_access.cidr"

wernerfred commented 3 years ago

It would be great if you provide the complete information that is asked by the Issue template as well as choosing the right Issue template. Imho the BUG template would have been the better choice. It asks for reproducible bugs and if you can reproduce it with certain settings we can look at it and try to help.

Otherwise it is 50/50 if we can find a solution for you :)

I have no clue about your environment but could it be that the mailserver sits behind a proxy? Either on the host itself or with dedicated hardware? I run into similar issues using proxy protocol behind traefik as the origin of the request is always the proxy despite showing the real IP in logs.

BlockI0tChain commented 3 years ago

Postscreen/F2B Issue => F2B doesn't block spambots:

Context

Understood, please have a look at this:

What is affected by this bug?

Mailserver is getting hammered by any prodding spambot. Every connection attempt seems to be logged, however without any reaction whatsoever from the F2B.

When does this occur?

24/7

How do we replicate the issue?

  1. I use the following stack with docker-compose 3.8 :
services:
  #Blk0
    Reverse-Proxy: networks(internet, local)
  #Certbot
     Certbot:networks(local)
  #BLK2
     Frontend0 (nginx-based): networks(local)
     Frontend1 (nginx-based): networks(local
  #Blk3
     Backend0 (debian): networks(local)
     Backend1 (debian): networks(local)
  #Blk4
     Mailserver (docker-mailserver:latest): ports(25,143,587,993,465), env_file(mailserver.env), cap_add(NET_ADMIN, SYS_PTRACE), networks(internet)

   networks: internet(external), local(driver: bridge)
  1. The Reverse Proxy service listen to 80, 443
  2. Mailserver is not behind the Proxy service and listens to: 25, 143, 587, 993, 465"

Behavior

Actual Behavior

Expected Behavior

Your Environment

Environment Variables


SPOOF_PROTECTION=1
ENABLE_FAIL2BAN=1
SSL_TYPE=letsencrypt

Relevant Stack Traces

# BEGIN  got lots of this sort:
mailserver       | Feb 15 19:31:21 mail postfix/postscreen[10972]: CONNECT from [87.246.7.226]:40982 to [192.168.176.2]:25
mailserver       | Feb 15 19:31:21 mail postfix/postscreen[10972]: cache btree:/var/lib/postfix/postscreen_cache full cleanup: retained=26 dropped=3 entries
mailserver       | Feb 15 19:31:22 mail postfix/dnsblog[10976]: addr 87.246.7.226 listed by domain list.dnswl.org as 127.0.10.3
mailserver       | Feb 15 19:31:22 mail postfix/dnsblog[10975]: addr 87.246.7.226 listed by domain bl.mailspike.net as 127.0.0.2
mailserver       | Feb 15 19:31:22 mail postfix/postscreen[10972]: PASS NEW [87.246.7.226]:40982
mailserver       | Feb 15 19:31:22 mail postfix/smtpd[10984]: warning: hostname net6-ip226.linkbg.com does not resolve to address 87.246.7.226: Name or service not known
mailserver       | Feb 15 19:31:22 mail postfix/smtpd[10984]: connect from unknown[87.246.7.226]
mailserver       | Feb 15 19:31:24 mail dovecot: auth: passwd-file(ns2@blkchaintech.de,87.246.7.226): unknown user (SHA1 of given password: e15e2f)
mailserver       | Feb 15 19:31:26 mail postfix/smtpd[10984]: warning: unknown[87.246.7.226]: SASL LOGIN authentication failed: UGFzc3dvcmQ6
mailserver       | Feb 15 19:31:26 mail postfix/smtpd[10984]: disconnect from unknown[87.246.7.226] ehlo=1 auth=0/1 rset=1 quit=1 commands=3/4
mailserver       | Feb 15 19:32:07 mail postfix/postscreen[10972]: CONNECT from [87.246.7.226]:49306 to [192.168.176.2]:25
mailserver       | Feb 15 19:32:07 mail postfix/postscreen[10972]: PASS OLD [87.246.7.226]:49306
mailserver       | Feb 15 19:32:07 mail postfix/smtpd[10984]: warning: hostname net6-ip226.linkbg.com does not resolve to address 87.246.7.226: Name or service not known
mailserver       | Feb 15 19:32:07 mail postfix/smtpd[10984]: connect from unknown[87.246.7.226]
mailserver       | Feb 15 19:32:11 mail dovecot: auth: passwd-file(darwin@blkchaintech.de,87.246.7.226): unknown user (SHA1 of given password: 0c0655)
mailserver       | Feb 15 19:32:13 mail postfix/smtpd[10984]: warning: unknown[87.246.7.226]: SASL LOGIN authentication failed: UGFzc3dvcmQ6
mailserver       | Feb 15 19:32:13 mail postfix/smtpd[10984]: disconnect from unknown[87.246.7.226] ehlo=1 auth=0/1 rset=1 quit=1 commands=3/4
mailserver       | Feb 15 19:32:54 mail postfix/postscreen[10972]: CONNECT from [87.246.7.226]:57546 to [192.168.176.2]:25
mailserver       | Feb 15 19:32:54 mail postfix/dnsblog[10977]: addr 87.246.7.226 listed by domain bl.mailspike.net as 127.0.0.2
mailserver       | Feb 15 19:32:54 mail postfix/dnsblog[10974]: addr 87.246.7.226 listed by domain list.dnswl.org as 127.0.10.3
mailserver       | Feb 15 19:32:54 mail postfix/postscreen[10972]: PASS OLD [87.246.7.226]:57546
mailserver       | Feb 15 19:32:54 mail postfix/smtpd[10984]: warning: hostname net6-ip226.linkbg.com does not resolve to address 87.246.7.226: Name or service not known
mailserver       | Feb 15 19:32:54 mail postfix/smtpd[10984]: connect from unknown[87.246.7.226]
mailserver       | Feb 15 19:32:59 mail dovecot: auth: passwd-file(net@blkchaintech,87.246.7.226): unknown user (SHA1 of given password: 976950)
mailserver       | Feb 15 19:33:01 mail postfix/smtpd[10984]: warning: unknown[87.246.7.226]: SASL LOGIN authentication failed: UGFzc3dvcmQ6
mailserver       | Feb 15 19:38:01 mail postfix/smtpd[10984]: timeout after AUTH from unknown[87.246.7.226]
mailserver       | Feb 15 19:38:01 mail postfix/smtpd[10984]: disconnect from unknown[87.246.7.226] ehlo=1 auth=0/1 rset=1 commands=2/3
mailserver       | Feb 15 19:41:21 mail postfix/anvil[10986]: statistics: max connection rate 2/60s for (smtpd:87.246.7.226) at Feb 15 18:32:07
mailserver       | Feb 15 19:41:21 mail postfix/anvil[10986]: statistics: max connection count 1 for (smtpd:87.246.7.226) at Feb 15 18:31:22
mailserver       | Feb 15 19:41:21 mail postfix/anvil[10986]: statistics: max cache size 1 at Feb 15 18:31:22
# END
wernerfred commented 3 years ago

quick question: as the domain is listed during dnsbl lookups you should be able to block them via postscreen option drop.

can you provide additional information e.g. fail2ban config or iptables -S from inside the container?

EDIT: just seen that you proposed postscreen in one of your comments yourself.

I'm wondering what's the issue here. Fail2ban is working fine in my environment(s).

BlockI0tChain commented 3 years ago

Sure: f2b cfg:

# Fail2Ban main configuration file
#
# Comments: use '#' for comment lines and ';' (following a space) for inline comments
#
# Changes:  in most of the cases you should not modify this
#           file, but provide customizations in fail2ban.local file, e.g.:
#
# [Definition]
# loglevel = DEBUG
#

[Definition]

# Option: loglevel
# Notes.: Set the log level output.
#         CRITICAL
#         ERROR
#         WARNING
#         NOTICE
#         INFO
#         DEBUG
# Values: [ LEVEL ]  Default: ERROR
#
loglevel = INFO

# Option: logtarget
# Notes.: Set the log target. This could be a file, SYSLOG, STDERR or STDOUT.
#         Only one log target can be specified.
#         If you change logtarget from the default value and you are
#         using logrotate -- also adjust or disable rotation in the
#         corresponding configuration file
#         (e.g. /etc/logrotate.d/fail2ban on Debian systems)
# Values: [ STDOUT | STDERR | SYSLOG | SYSOUT | FILE ]  Default: STDERR
#
logtarget = /var/log/fail2ban.log

# Option: syslogsocket
# Notes: Set the syslog socket file. Only used when logtarget is SYSLOG
#        auto uses platform.system() to determine predefined paths
# Values: [ auto | FILE ]  Default: auto
syslogsocket = auto

# Option: socket
# Notes.: Set the socket file. This is used to communicate with the daemon. Do
#         not remove this file when Fail2ban runs. It will not be possible to
#         communicate with the server afterwards.
# Values: [ FILE ]  Default: /var/run/fail2ban/fail2ban.sock
#
socket = /var/run/fail2ban/fail2ban.sock

# Option: pidfile
# Notes.: Set the PID file. This is used to store the process ID of the
#         fail2ban server.
# Values: [ FILE ]  Default: /var/run/fail2ban/fail2ban.pid
#
pidfile = /var/run/fail2ban/fail2ban.pid

# Options: dbfile
# Notes.: Set the file for the fail2ban persistent data to be stored.
#         A value of ":memory:" means database is only stored in memory
#         and data is lost when fail2ban is stopped.
#         A value of "None" disables the database.
# Values: [ None :memory: FILE ] Default: /var/lib/fail2ban/fail2ban.sqlite3
dbfile = /var/lib/fail2ban/fail2ban.sqlite3

# Options: dbpurgeage
# Notes.: Sets age at which bans should be purged from the database
# Values: [ SECONDS ] Default: 86400 (24hours)
dbpurgeage = 1d

Iptables is however populated by the bans I manually jailed using f2b-client into the postfix-sasl jail :


root@mail:/etc/fail2ban# iptables -S
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-N f2b-postfix-sasl
-A INPUT -p tcp -m multiport --dports 25,465,587,143,993,110,995 -j f2b-postfix-sasl
-A f2b-postfix-sasl -s 193.169.255.216/32 -j REJECT --reject-with icmp-port-unreachable
... and this goes on ...
-A f2b-postfix-sasl -s 167.248.133.54/32 -j REJECT --reject-with icmp-port-unreachable
-A f2b-postfix-sasl -s 162.142.125.56/32 -j REJECT --reject-with icmp-port-unreachable
-A f2b-postfix-sasl -s 162.142.125.39/32 -j REJECT --reject-with icmp-port-unreachable
-A f2b-postfix-sasl -s 139.59.226.224/32 -j REJECT --reject-with icmp-port-unreachable
-A f2b-postfix-sasl -s 128.14.134.134/32 -j REJECT --reject-with icmp-port-unreachable
-A f2b-postfix-sasl -j RETURN
casperklein commented 3 years ago

@docker-mailserver/maintainers

I noticed similar in the past, that fail2ban seems not to recognize all "bad" things. Maybe the whole fail2ban thing should be reviewed/refactored/updated. For example, why do we exactly need this: https://github.com/docker-mailserver/docker-mailserver/blob/master/target/fail2ban/jail.conf

It looks like the default config from three years ago. In the meantime, f2b was updated and also the default config was tweaked. If we need some custom settings, only these should go into e.g. fail2ban.local instead of copying the default config and modifying it. From the man page:

# Changes: in most of the cases you should not modify this file, but provide customizations in fail2ban.local file

georglauterbach commented 3 years ago

@casperklein Actually, now that you say it, I've been seeing this in my logs (from time to time) too. Could you have a look at it?

polarathene commented 3 years ago

Maybe the whole fail2ban thing should be reviewed/refactored/updated.

I have identified similar concerns with this PR that got closed during migration. I don't have time at present to work on it, maybe next month.

casperklein commented 3 years ago

I've just opened #1821, to track/discuss the further proceeding.

georglauterbach commented 3 years ago

Closing dues to processing in #1821.

casperklein commented 3 years ago

BlockI0tChain wrote:

My report as follows:

Used the same env as in #1821 with the lastest master#4127370:

Jail config :

Can't put my finger on it yet. I suspect something is ill-set. All I can say for sure is that the whole cascade of the filter failures logged in the fail2ban status of the jails will most probably account for the ban enforcement issues.

Please let me know if you can see something in the logs:

casperklein commented 3 years ago

I've re-read the whole issue. I don't think, there is anything wrong with the fail2ban implementation in this project. There isn't much magic done. It's just a default Debian package installation, with a some configuration options to make things easier.

Now some of the bots are getting banned, most of them however aren't!!

If some bot actions are not recognized/banned, then probably there is no filter for that in the default configuration. We don't provide our own filters. However, we are not using the latest version of fail2ban, but the latest version available in the Debian repository. You could try a newer fail2ban version and see if that improve things. Remember, fail2ban specific questions (not matching filters etc), should be placed at best in the upstream project. Don't get me wrong, if you have an evidence, that your problem is specific to docker-mailserver, we will try to fix that.

I don't know if you forgot to paste it, but the first line in config/fail2ban-jail.cf should be [DEFAULT]

casperklein commented 3 years ago

I think what you are looking for is a "fail2ban postscreen" filter. I found this, which appears identical to your problem: http://postfix.1071664.n5.nabble.com/postscreen-fail2ban-filter-td91317.html

BlockI0tChain commented 3 years ago

@casperklein Thanks for your considerations, I believe as well that the matching filters configuration is the culprit here. If however the F2B filtering won't kick in even for the failed brute force authentications requests then I'm afraid the service isn't really "healthy". You can clearly see this in the log when many brute force connections occur and those IPs won't get banned. I understand this isn't a docker-mailserver specific issue, tho Imo that's a serious security issue for any mailserver out there.

If we look at the postfix filter set in the "aggressive mode" this should ban any IP implicated in:

I tested both of these expressions on the posted log text and and it does matches all those lines flawlessly. Except the banning doesn't occur, none of those IP with matched requests are banned in postfix jail nor in the postfix-sasl jail.

I ain't sure about this however why should be there a postfix-sasl jail enabled that's supposed to do exactly what the auth/auth2 rules on the postfix[mode = aggressive] already do? Or I'm missing something else here?
Dunno wherefrom is the fail2ban loading the filters for those spawned jails as it won't say anything in the fail2ban.log so I assumed from the "filter.d" folder, however I've noticed that there's no postfix-sasl.conf file in the "./fail2ban/filter.d" folder. Now I wonder how can we know what filters is fail2ban actually using for those jails? I reckon this would help to fix the issue.

Nevertheless I'll have a look at your suggestion and some other solutions to update the filters. Would be good to know tho how to set those filters the right way too.

georglauterbach commented 3 years ago

You can clearly see this in the log when many brute force connections occur and those IPs won't get banned. I understand this isn't a docker-mailserver specific issue, tho Imo that's a serious security issue for any mailserver out there.

If we look at the postfix filter set in the "aggressive mode" this should ban any IP implicated in:

* any failed authentication logged in the form "SASL LOGIN authentication failed"; here the mdre-auth2 regex should
  easily match the lines in the log,

* any ddos requests logged as "lost connection after"; here the ddos regex would as well match the lines in the log

I see the same behavior - and was wondering if this is normal. I couldn't image this being normal, and I asked about this (maybe in a complicated fashion) earlier. I'm seeing these LOGIN authentication failed and lost connection after many times and these should be banned. I'm using mode = aggressive too, but it does not seem to do anything really...

I tested both of these expressions on the posted log text and and it does matches all those lines flawlessly. Except the banning doesn't occur, none of those IP with matched requests are banned in postfix jail nor in the postfix-sasl jail.

Can confirm.


I will switch to :edge in a few minutes and check back if this is still the case.

casperklein commented 3 years ago

I ain't sure about this however why should be there a postfix-sasl jail enabled that's supposed to do exactly what the auth/auth2 rules on the postfix[mode = aggressive] already do?

Because we don't use mode = aggressive by default.

These are the only settings we changed, compared to the default debian configuration of fail2ban: https://github.com/docker-mailserver/docker-mailserver/blob/master/target/fail2ban/jail.local

To rule DMS out, what you can do is:

Install (maybe latest) fail2ban on the host system and point the fail2ban logpaths to your mailserver mail.log.

What bugs me personal, is the now three year old fail2ban version (0.10.2) included in Debian :worried: Debian 11 will include the current release 0.11.2.

You can clearly see this in the log when many brute force connections occur and those IPs won't get banned.

Maybe you can quote the exact example, but I think this is what I meant with missing postscreen filter

BlockI0tChain commented 3 years ago

Maybe you can quote the exact example, but I think this is what I meant with missing postscreen filter

I guess my log is not that long, but have a look at these mdpr-auth warnings taken just now from my log. The IP 31.210.20.93 isn't banned whatsoever. It just registers as 'failed' in the postfix jail status What I do now as a workaround: I'm extra parsing the mail.log with a script and ban those "bรถse" IPs.

casperklein commented 3 years ago

I see 3 times [31.210.20.93]: SASL LOGIN authentication failed:

However in a time range > 10m. Your findtime is configured to 10m. You could try to increase findtime to something greater, e.g. 1h or 1d.

BlockI0tChain commented 3 years ago

More than "maxretry" occurrences shouldn't be allowed tho and that IP had other recurrent events afterwards. Also increasing the "findtime" doesn't seem to have any effect.

casperklein commented 3 years ago

More than "maxretry" occurrences shouldn't be allowed tho

That is not correct. From the manual A host is banned if it has generated "maxretry" during the last "findtime" seconds.

If findtime is set to 10m, an attacker can make a login attempt every 11th minute and won't get banned ever.

Also increasing the "findtime" doesn't seem to have any effect.

Just to be sure: Don't forget to restart fail2ban/DMS after making changes to the configuration.

TBH I am out of ideas. If even the approach with the latest fail2ban version on the host does not resolve the problem, I suggest opening an issue upstream.

casperklein commented 3 years ago

Works like a charm on my side. Three auth failures, then banned.

Apr 20 19:16:16 mail postfix/postscreen[4459]: CONNECT from [61.168.104.180]:38263 to [172.26.0.2]:25
Apr 20 19:16:22 mail postfix/postscreen[4459]: PASS NEW [61.168.104.180]:38263
Apr 20 19:16:26 mail postfix/smtpd[4464]: warning: hostname pc180.zz.ha.cn does not resolve to address 61.168.104.180: Name or service not known
Apr 20 19:16:26 mail postfix/smtpd[4464]: connect from unknown[61.168.104.180]
Apr 20 19:16:27 mail dovecot: auth: passwd-file(nologin,61.168.104.180): unknown user (SHA1 of given password: 816a33)
Apr 20 19:16:29 mail postfix/smtpd[4464]: warning: unknown[61.168.104.180]: SASL LOGIN authentication failed: UGFzc3dvcmQ6
Apr 20 19:16:29 mail postfix/smtpd[4464]: disconnect from unknown[61.168.104.180] ehlo=1 auth=0/1 quit=1 commands=2/3

Apr 20 19:16:34 mail postfix/postscreen[4459]: CONNECT from [61.168.104.180]:38936 to [172.26.0.2]:25
Apr 20 19:16:34 mail postfix/postscreen[4459]: PASS OLD [61.168.104.180]:38936
Apr 20 19:16:34 mail postfix/smtpd[4464]: warning: hostname pc180.zz.ha.cn does not resolve to address 61.168.104.180: Name or service not known
Apr 20 19:16:34 mail postfix/smtpd[4464]: connect from unknown[61.168.104.180]
Apr 20 19:16:39 mail dovecot: auth: passwd-file(isaiah@example.com,61.168.104.180): unknown user (SHA1 of given password: 50f749)
Apr 20 19:16:41 mail postfix/smtpd[4464]: warning: unknown[61.168.104.180]: SASL LOGIN authentication failed: UGFzc3dvcmQ6
Apr 20 19:16:41 mail postfix/smtpd[4464]: disconnect from unknown[61.168.104.180] ehlo=1 auth=0/1 quit=1 commands=2/3

Apr 20 19:16:47 mail postfix/postscreen[4459]: CONNECT from [61.168.104.180]:39476 to [172.26.0.2]:25
Apr 20 19:16:47 mail postfix/postscreen[4459]: PASS OLD [61.168.104.180]:39476
Apr 20 19:16:47 mail postfix/smtpd[4464]: warning: hostname pc180.zz.ha.cn does not resolve to address 61.168.104.180: Name or service not known
Apr 20 19:16:47 mail postfix/smtpd[4464]: connect from unknown[61.168.104.180]
Apr 20 19:16:56 mail dovecot: auth: passwd-file(isaiah,61.168.104.180): unknown user (SHA1 of given password: 50f749)
Apr 20 19:16:58 mail postfix/smtpd[4464]: warning: unknown[61.168.104.180]: SASL LOGIN authentication failed: UGFzc3dvcmQ6
Apr 20 19:16:58 mail postfix/smtpd[4464]: disconnect from unknown[61.168.104.180] ehlo=1 auth=0/1 quit=1 commands=2/3
docker exec mailserver fail2ban | grep 61.168.104.180
Banned in postfix-sasl: 61.168.104.180, [..]
docker exec mailserver iptables -L -n | grep 61.168.104.180
DROP       all  --  61.168.104.180       0.0.0.0/0
BlockI0tChain commented 3 years ago

I guess that's what I also meant to convey with the "maxretry" , as in "maxretry" per "findtime". Problem is that I have changed it to 1d and no ban occurred after 2 or more consecutive auth attempts :( I appreciate you taking the time and I'm glad it works fine for you. I'm pretty sure I'll fix mine as well soon.

On other note I found out why isn't there any postfix-sasl.conf in filters.d to be found:

So postfix-sasl is already configured with filter = postfix[mode=auth] in "./fail2ban/jail.conf"

As I understand a custom jail would take:

# Custom Fail2Ban filter configuration [Definition] _daemon = failregex = ignoreregex =

# Custom Fail2Ban jail configuration [custom-jail] enabled = true filter = custom-filter logpath = maxretry = findtime = bantime =

I'll give it a go in finding a good postscreen filter. Hope it'll work.

casperklein commented 3 years ago

I'm myself pretty new to f2b. I started getting familiar with it some weeks ago, when working on the cleanup PR. That said, I've never created own filters in f2b, but what you wrote looks good to me. Please report back, if you find a solution for your problem.

Beside that, did you test with fail2ban 0.11.2? There were lots of changes/fixes since "our" used version. Disclaimer:

0.11 is totally compatible to 0.10 (configuration- and API-related stuff), but the database
    got some new tables and fields (auto-converted during the first start), so once updated to 0.11, you
    have to remove the database /var/lib/fail2ban/fail2ban.sqlite3 (or its different to 0.10 schema)
    if you would need to downgrade to 0.10 for some reason.
georglauterbach commented 3 years ago

@BlockI0tChain Have you had success with Postscreen or any other method? I'd be interested in it since I experience the same "problem".

BlockI0tChain commented 3 years ago

@casperklein I've tested 0.11 on the host and it flawlessly banned all the "auth failures". I've planned to "cook" a custom docker-mailserver image with the 0.11 but haven't find the time to work on it yet.

@aendeavor It's hacky but it somehow works, I've set "findtime" and the "bantime" to 10d and it seems to did the trick with the "auth failures", every attempt lead to ban, with only a few failures.
As for the postscreen noise filter I've added:

And no more clutter in the logs. You can give it a try! Hope it works.

casperklein commented 3 years ago

Same here, I am planning to use 0.11 in my personal build too, because there is still no ETA when Debian 11 will be released.

BlockI0tChain commented 3 years ago

Cool, I'll post the docker file if I'll manage to get it done.

georglauterbach commented 3 years ago

Thanks!

BlockI0tChain commented 3 years ago

One important aspect I forgot to mention: The regex line in the postscreen.conf: ^%(__prefix_line)saddr <HOST> listed by domain .* as .*$ is problematic since it might ban some legit server if it fails (which happens quite often due to various reasons) to connect after "maxretry" within your set 'bantime'.
So I'd suggest to maybe leave it out.

georglauterbach commented 3 years ago

Same here, I am planning to use 0.11 in my personal build too, because there is still no ETA when Debian 11 will be released.

Please share this with us when you're done - I'm interested :D


One important aspect I forgot to mention: The regex line in the postscreen.conf: ^%(__prefix_line)saddr listed by domain . as .$ is problematic since it might ban some legit server if it fails (which happens quite often due to various reasons) to connect after "maxretry" within your set 'bantime'. So I'd suggest to maybe leave it out.

Alright, I will leave this out.

casperklein commented 3 years ago

The installation is pretty easy, there is a debian package available: https://github.com/fail2ban/fail2ban/releases/download/0.11.2/fail2ban_0.11.2-1.upstream1_all.deb

georglauterbach commented 3 years ago

This package depends on python3:any and lsb-base. I would need to run apt-get update and so on and do all the stuff the Dockerfile does to remove image size again. Any improvements over my current approach? :)

georglauterbach commented 3 years ago

Maybe we should consider installing F2B this way in the Dockerfile itself (?)

casperklein commented 3 years ago

python3 and lsb-base are installed as dependency by other packages already. This should be enough:

48c48
<   ed fail2ban fetchmail file gamin gnupg gzip iproute2 iptables \
---
>   ed fetchmail file gamin gnupg gzip iproute2 iptables \
56a57,60
>   # Fail2ban
>   curl -Lso fail2ban.deb https://github.com/fail2ban/fail2ban/releases/download/0.11.2/fail2ban_0.11.2-1.upstream1_all.deb && \
>   dpkg -i fail2ban.deb && \
>   rm fail2ban.deb && \

Maybe we should consider installing F2B this way in the Dockerfile itself (?)

Yes. What were you talking about? ๐Ÿ˜„ user-patches.sh?

georglauterbach commented 3 years ago

python3 and lsb-base are installed as dependency by other packages already. This should be enough:


48c48

<   ed fail2ban fetchmail file gamin gnupg gzip iproute2 iptables \

---

>   ed fetchmail file gamin gnupg gzip iproute2 iptables \

56a57,60

>   # Fail2ban

>   curl -Lso fail2ban.deb https://github.com/fail2ban/fail2ban/releases/download/0.11.2/fail2ban_0.11.2-1.upstream1_all.deb && \

>   dpkg -i fail2ban.deb && \

>   rm fail2ban.deb && \

Maybe we should consider installing F2B this way in the Dockerfile itself (?)

Yes. What were you talking about? ๐Ÿ˜„ user-patches.sh?

Ahh, I only checked debian:buster-slim ... And yes, I was in user-patches.sh :D Will you create a PR?

casperklein commented 3 years ago

I am all for it. But we had similar discussions already (e.g. if we should use the more recent Dovecot community repo) and decided to stick with the older Debian provided packages.

georglauterbach commented 3 years ago

I am all for it. But we had similar discussions already (e.g. if we should use the more recent Dovecot community repo) and decided to stick with the older Debian provided packages.

I see. I'd nevertheless do it if you ask me. If you want to, we could open a small maintainers discussion first.

casperklein commented 3 years ago

I've to correct myself, the dovecot community repo is not used, because it doesn't provide ARM packages. We had then a little discussion about keeping it commented in the Dockerfile or not.

georglauterbach commented 3 years ago

@BlockI0tChain When you're using F2B 0.11, do you still need the "hacks" and the Postscreen filter?

BlockI0tChain commented 3 years ago

While the "auth failures" were filtered out of the box with the mode = aggressive in the postfix config, the Postscreen filter is still needed. I've tested this on the host so I think this still need to be duration-tested in the docker though.

georglauterbach commented 3 years ago

While the "auth failures" were filtered out of the box with the mode = aggressive in the postfix config, the Postscreen filter is still needed.

I've tested this on the host so I think this still need to be duration-tested in the docker though.

And do you still need 10d ban- and findtime?

BlockI0tChain commented 3 years ago

That's what I had defined in configs and unfortunately I can't say since I haven't changed them, so this needs testing as well.

casperklein commented 3 years ago

The edge image now contains fail2ban 0.11.2 ๐ŸŽ‰

BlockI0tChain commented 3 years ago

Awesome job! Thank you guys!