question: Why does Fail2Ban not respect my whitelist `ignoreip` setting?

[MarcS1975]

📝 Preliminary Checks

• [X] I tried searching for an existing issue and followed the debugging docs advice, but still need assistance.

👀 What Happened?

Fail2ban whitelisting does not work. I am trying ignore any lockups caused by IMAP clients hosted on my own network but this doesn't work. When I execute several 'incorrect' login attempts , my external IP still gets blocked by fail2ban (->banned in dovecot).

In accordance with the documentation I have added in the file fail2ban-jail.cfg stored in the DMS 'config' folder ignoreip = 127.0.0.1/8,xx.xx.xx.xxx whereby xx.xx.xx.xxx = my external IP4

But my IP does not get ignored by fail2ban and still gets blocked.

👟 Reproduction Steps

log into mail client (Roundcube or Rainloop) that connects to DMS use wrong password mail client IP gets blocked

🐋 DMS Version

13.3.1

💻 Operating System and Architecture

Debian 11

⚙️ Container configuration files

mailserver:
    image: docker.io/mailserver/docker-mailserver:13.3.1
    container_name: mailserver2
    # Provide the FQDN of your mail server here (Your DNS MX record should point to this value)
    hostname: [myhost]
    ports:
      [myports]
    volumes:
    - /[mypath]/mail-data/:/var/mail/
    - /[mypath]/mail-state/:/var/mail-state/
    - /[mypath]/mail-logs/:/var/log/mail/
    - /[mypath]/config/:/tmp/docker-mailserver/
    - /etc/localtime/:/etc/localtime:ro
    - /[mypath]/certs/:/tmp/dms/custom-certs/

    environment:
      - TZ=Europe/London
      - LOG_LEVEL=debug 
      - PFLOGSUMM_TRIGGER=daily_cron # by AJ
      - POSTFIX_MESSAGE_SIZE_LIMIT=10240000 # by AJ
      - LOGWATCH_INTERVAL=daily 
      - ENABLE_SPAMASSASSIN=1
      - ENABLE_SPAMASSASSIN_KAM=0 
      - SPAMASSASSIN_SPAM_TO_INBOX=1
      - ENABLE_CLAMAV=0 # by AJ
      - ENABLE_FAIL2BAN=1
      - ENABLE_POSTGREY=0
      - SSL_TYPE=manual
      - SSL_CERT_PATH=/tmp/dms/custom-certs/fullchain.pem
      - SSL_KEY_PATH=/tmp/dms/custom-certs/privkey.pem
    restart: always
    stop_grace_period: 1m
    cap_add:
      - NET_ADMIN
 # healthcheck:
 #  test: "ss --listening --tcp | grep -P 'LISTEN.+:smtp' || exit 1"
 # timeout: 3s
 # retries: 0

📜 Relevant log output

No response

Improvements to this form?

No response

[polarathene]

log into mail client (Roundcube or Rainloop) that connects to DMS

Your not logging to DMS directly from a mail client, your IP is going to be logged as the IP for Roundcube/Rainloop (assuming this is a container too). You need that webmail service to have a way to provide the real client IP to DMS.

There has been past discussions and issues on this topic, you can try search for those for more insights.

@casperklein may know of better advice but AFAIK since Fail2Ban is operating within the DMS container and only monitors logs of services we run, Fail2Ban is only going to see the IP from the connecting client (Roundcube/Rainloop), not the actual user IP, thus it cannot function in the way you'd expect.

You'll probably need to run Fail2Ban outside of DMS so that it can monitor logs from those webmail services you're using instead, and ban the client connection at that point. No point in trying to have Fail2Ban in DMS monitor those logs, as it will only receive the IP connection from Roundcube/Rainloop, that's all you can ban on which allows a malicious user to trigger a ban and all users of that service become banned as a result.

[MarcS1975]

OK Lets ignore Rainloop for the moment and just focus on Roundcube. If I am logging into Roundcube, that will then automatically log into DMS IMAP to get my emails. Yes, Roundcube runs in a container but that is irrelevant. Roundcube uses my DMS URL and logs in with its own external IP address. And that external IP is in fail2ban's whitelist but does not work. What am I getting wrong?

User->Roundcube--ext_IP1----> DMS---fail2ban (sees Roundcube's external IP)

[polarathene]

Roundcube uses my DMS URL and logs in with its own external IP address.

How are you verifying this?

If they run on the same system Docker may route it more directly rather than redundantly through separate interfaces when it realizes it doesn't need to.

Sometimes that loses the original IP of the client (Roundcube in this case) and replaces it with the gateway IP (or the container IP, if you have not enforced the request to go through your external IP, other interfaces available to the container are valid too). I documented a table for this behaviour that you can refer to for more info (it's not just for IPv6 IIRC).

And that external IP is in fail2ban's whitelist but does not work. What am I getting wrong?

Have you confirmed that is the IP in the logs? What happens if you disable Fail2Ban? Are connections suddenly successful?

Is this external IP an IPv6 address? Nevermind you said IPv4

---

      - SSL_TYPE=manual
      - SSL_CERT_PATH=/tmp/dms/custom-certs/fullchain.pem
      - SSL_KEY_PATH=/tmp/dms/custom-certs/privkey.pem

This might also be an issue for your separate Roundcube container, if these are not LetsEncrypt, but your own custom/local CA, you also need to ensure Roundcube has the CA public cert in it's trust store to successfully verify TLS connections to DMS.

[MarcS1975]

How are you verifying this?

I can see that it works. My emails are there and I have set RC up with my domain name of the mailserver. It will have to go via the DNS to fetch the ext IP. The RC email access is not the problem. I can access all my emails fine. Its just when I have a case of incorrect password (manually caused by my mistyping) then fail2ban blocks the RC IP. I can see that happening on the DSM fail2ban logs, when my IP appears. The problem is that DMS-fail2ban then ignores my whitelist. It should not block my external IP (even if I mistype the password 100x) but fail2ban treats RC login attempts just like any normal intruder.

Letsencrypt is also not an issue. As I said login into DSM via RC and reading my emails works 100%.

[polarathene]

I have set RC up with my domain name of the mailserver. It will have to go via the DNS to fetch the ext IP.

Just because you resolve the IP to connect to, does not mean the request from RC is made through external IP network interface, could be container to container if they're both connected to a shared network interface. That is why I asked about verification with the logs.

In the link I provided, I had done tests in the past to connect from one container to another indirectly through the host DNS which would resolve to the host external IP... or internal network IP of requesting container as the source IP address.

Its just when I have a case of incorrect password (manually caused by my mistyping) then fail2ban blocks the RC IP. I can see that happening on the DSM fail2ban logs, when my IP appears.

That's what I was talking about. If the same external IP is logged and banned, then Fail2Ban is doing it's thing blocking RC.

The problem is that DMS-fail2ban then ignores my whitelist. It should not block my external IP (even if I mistype the password 100x) but fail2ban treats RC login attempts just like any normal intruder.

If you can confidently verify that IP in the logs being blocked is the same one that you have in your ignore list, ok that sounds like an issue. Although I don't think I've seen anyone else report this yet so I'm not sure why it affects you.

If RC is the only way to authenticate to DMS, why do you even need Fail2Ban running? Just don't expose the mail ports to a public interface if RC is sufficient?

---

You should probably also verify this issue still happens with v14 due to all the updates there, it's possible that any bug you're experiencing may have already been fixed upstream.

[MarcS1975]

could be container to container if they're both connected to a shared network interface. T

nope. my setup is not that sophisticated. All separate Docker networks so they do go via external DNS. Apart from that, I can see my external IP appearing in the fail2ban logs.

If the same external IP is logged and banned, then Fail2Ban is doing it's thing blocking RC.

But that's what the whitelist entry is for, isn't it. To tell fail2ban "Do not ban this IP even if it is breaching the fail2ban rules (5x failed login).

If you can confidently verify that IP in the logs being blocked is the same one that you have in your ignore list,

Yes thats my point. It is the same as I put it in there and I know my external IP :)

[casperklein]

In accordance with the documentation I have added in the file fail2ban-jail.cfg stored in the DMS 'config' folder

Is this a typo? It should be fail2ban-jail.cf.

I just gave it a try and the ignore feature works as expected for me:

/var/log/fail2ban.log:

2024-07-09 21:38:10,520 fail2ban.filter         [2350]: INFO    [dovecot] Ignore XXX.XXX.XXX.XXX by ip

ignoreip = 127.0.0.1/8,xx.xx.xx.xxx

I use a space instead of the comma, but that shouldn't be an issue. According to fail2ban, both is valid.

fyi: The Fail2ban package was updated after v14. The current edge build contains the latest fail2ban version. You may also give this a try.

[MarcS1975]

Is this a typo? It should be fail2ban-jail.cf.

Damn! This is probably the issue. I created rules but used the wrong filename. Thanks for pointing that out! My stupid mistake. I will do some more testing but I am sure this was the issue. The cf file in the config was simply ignored by DMS.

To prevent this, would it be possible when DMS is launching to write a line in the log file for each of the customised .cf files when it is read by DMS? I can watch the log with Dozzle like below...

09:46:41 PM
[   INF   ]  Starting daemons
07/09/2024
09:46:41 PM
[  DEBUG  ]  Starting cron
07/09/2024
09:46:42 PM
[  DEBUG  ]  Starting rsyslog
07/09/2024
09:46:42 PM
[  DEBUG  ]  Starting dovecot
07/09/2024
09:46:43 PM
[  DEBUG  ]  Starting update-check
07/09/2024
09:46:43 PM
[  DEBUG  ]  Starting opendkim
07/09/2024
09:46:44 PM
[  DEBUG  ]  Starting opendmarc
07/09/2024
09:46:45 PM
[  DEBUG  ]  Starting postfix
07/09/2024
09:46:45 PM
[  DEBUG  ]  Starting fail2ban
07/09/2024
09:46:46 PM
[  DEBUG  ]  Starting amavis
07/09/2024
09:46:47 PM
[  DEBUG  ]  Starting changedetector
07/09/2024
09:46:47 PM
[   INF   ]  [myDomain] is up and running

[MarcS1975]

You may also give this a try.

I would love to upgrade to v14 but the DMS changelog currently warns This update alone involves **breaking changes** ... so is it really advised to upgrade to v14 or wait?

[polarathene]

so is it really advised to upgrade to v14 or wait?

Wait for what? Every major version release is breaking changes.

They sometimes don't apply to everyone, you need to read over the CHANGELOG.md for the release to see if any breaking change applies to you.

You will want to upgrade eventually, and regardless of how long you wait, you'll need to consider breaking changes across all major releases since the version you're on.

v14 is a big release, usually it's not so much to consider but we try to at least make it easier to digest by categorizing the scope of breaking changes. Some of them are just informational and may not cause you any problems, but they provide context when users do experience upgrade issues, where those problems may be coming from.

---

BTW in future when you experience such problems with a feature you try and you're new to a config file you've added, probably best to troubleshoot that the config file itself works by verifying at least some setting in the file can actually adjust behaviour, or reviewing the Fail2Ban docs we have.

I don't have much to do with the F2B support, so assumed you had that already sussed out, I didn't even consider the file name being misconfigured 😅 (thanks @casperklein !)

[polarathene]

write a line in the log file for each of the customised .cf files when it is read by DMS?

We do that with quite a few of the files actually. This includes Fail2Ban:

https://github.com/docker-mailserver/docker-mailserver/blob/4778f15fdaf5e94aee364396166fdc2821066c46/target/scripts/startup/setup.d/security/misc.sh#L192-L204

Follow our troubleshooting guide in future and use LOG_LEVEL=trace (the very first step we advise you to try) and you should already get that information.

[polarathene]

Closing as resolved.

[MarcS1975]

BTW in future when you experience such problems with a feature you try and you're new to a config file you've added, probably best to troubleshoot that the config file itself works by verifying at least some setting in the file can actually adjust behaviour, or reviewing the

Yes- thats exactly what I was doing here. I added IPs to my whitelist and checked if the desired behavior works. My own stupid mistake of course was the wrong name of the config file. Thanks anyway.

[polarathene]

Yes- thats exactly what I was doing here.

Kinda, but I was hinting more towards double checking with other settings in the config file to verify from feedback on those changes if Fail2Ban would actually reflect any change made. This issue was largely focused around the sole setting and questioning what else could be going wrong with the functionality, if it was a bug in DMS or environment related.

If you troubleshooted with the config by realizing that nothing in the config would have any effect, you would sooner question why that isn't working, and we'd arrive at the realization of the filename being wrong sooner :)

I make simple mistakes like that one all the time and sometimes don't realize until much later myself, so it's all good 👍 Glad it's sorted!

---

Perhaps we should add that to the troubleshooting docs page 🤷‍♂️