Fail2Ban fails to set the iptable rule

[dacianb]

Hello,

I've been experimenting with this, but for me it simply does not work. My setup: OS: Ubuntu 22.04 LTS Docker version 23.0.1, build a5ee5b1 Portainer CE 2.16.2

I deployed the docker-compose stack with Portainer and made sure all the log files are accessible and i tried to test the config by making a bunch of 404 requests to one of the NPM hosts. In the logs i can see that f2b reads the files and bans the IP, but nothing really happens. I tried to debug the issue, but was unable to do so. I tried changing the iptable chain to INPUT and FORWARD but still does not work. From what i can tell the f2b container does not comunicate with the docker host, and when i do a iptables -nvL on the docker host, there is no entry in the DOCKER-USER chain. I added the network_mode: "host" to the docker-compose yaml but still nothing. Docker is not running in rootless mode, everything is pretty much default docker installation.

Any ideea why this is happening?

Thanks in advance!

[l4rm4nd]

Sorry to hear about your inconvenience!

I am running the exact same docker-compose.yml productively with no issues so far. I am using also a Ubuntu OS but it's a virtual machine running on Proxmox VE hypervisor.

Let's compare the environment:

# cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04.5 LTS"

#docker --version
Docker version 23.0.0, build e92dd87

# iptables --version
iptables v1.8.4 (legacy)

Do you even have a DOCKER-USER chain?

[dacianb]

Wow, did not expect to receive such a quick response :smile: Well i am using the exact same setup (Proxmox VE with a docker VM). To answer your question, yes i can see the DOCKER-USER chain in the docker host VM, but nothing appears there. The only difference i see is that i am using Ubuntu 22.04 LTS and iptables is iptables v1.8.7 (nf_tables)

[l4rm4nd]

I highly assume that it's either your iptables version or the kernel that is not compiled to support string matching.

May read here: https://www.wiztelsys.com/Article_iptables_bob2.html

Since we are using a trick to ban IP addresses, string matching must be supported. The packets always originate from your reverse proxy, so iptables only sees the IP address of your proxy. Banning this IP is nonsense, so we must extract the real visitor's IP address from forwarded HTTP headers like X-Forwarded-For.

The trick is applied for example here.

Let's see an excerpt:

actionban = iptables -I DOCKER-USER -m string --algo bm --string 'X-Forwarded-For: <ip>' -j DROP

So make sure your kernel supports string matching and that your reverse proxy the packets are passing containing the real visitor IP in the X-Forwarded-For header.

An alternative fix would be to use Cloudflare. Then you can rely on action-ban-cloudflare.conf, which is more robust. If you go this way, ensure to configure a whitelist either at your reverse proxy or within Proxymox VE itself to only allow HTTP requests originating from Cloudflare's IP addresses listed here.

[dacianb]

Yeah, it seems that the problem is with nftables fw, since it's not 1:1 compatible with iptables. I found this mentioned here: https://unix.stackexchange.com/questions/655688/iptables-string-rules-translation-to-nftables

The fix would be to install legacy iptables or to try and implement the DROP statements with the nftables sintax.

Thanks for the help!

[l4rm4nd]

Also note that the jail.conf ignores IP addresses from private class subnets. So if you are testing a fail2ban ban locally and the logs are filled with private class IPs, there won't be an effective ban.

Consult the fail2ban logs. They will print offending IPs.

[dacianb]

Also note that the jail.conf ignores IP addresses from private class subnets. So if you are testing a fail2ban ban locally and the logs are filled with private class IPs, there won't be an effective ban.

Consult the fail2ban logs. They will print offending IPs.

Yes, i tested with a remote IP that is not in the ignore ip list. As i mentioned before, the banning itself works, i can see it in the f2b logs but the problem is iptables is not banning the ip :laughing:

[l4rm4nd]

Also note that the jail.conf ignores IP addresses from private class subnets. So if you are testing a fail2ban ban locally and the logs are filled with private class IPs, there won't be an effective ban. Consult the fail2ban logs. They will print offending IPs.

Yes, i tested with a remote IP that is not in the ignore ip list. As i mentioned before, the banning itself works, i can see it in the f2b logs but the problem is iptables is not banning the ip 😆

Alright, then it is the missing support for the iptables string extension in nftables.

[l4rm4nd]

implement the DROP statements with the nftables sintax

Would be hard to implement, as nftables do not support string matching.

This is the crux why we have to use legacy iptables with the string matching extension. The real visitor's IP address is hidden as client HTTP header within the network packet. Iptables or nftables natively only see the source IP address, which is usually your reverse proxy or Cloudflare. So you must have a way to inspect the packet and parse the X-Forwarded-For headers (respectively CF-Connecting-IP). This is done with the string matching extension, which nftables does not support.

[sinemeter]

Hi there, so same issue on my setup I guess. I use debian 11 on a Strato V-server.

$ sudo iptables --version iptables v1.8.7 (legacy)

Though debian does not explicitely say that its nf tables it is when I look at the /proc/modules file as described in the referenced article above (at least I see some entries there beginning with nftables... .

However in this case as my server is brand new with not much installed I would consider to switch to ubuntu as OS becasue Strato offers that also even as default setup.

Or does somebody know if there is another proxy manager that offers the IP banning feature out of the box - maybe does Traeffik do this?

[l4rm4nd]

Hi there, so same issue on my setup I guess. I use debian 11 on a Strato V-server.

$ sudo iptables --version iptables v1.8.7 (legacy)

Though debian does not explicitely say that its nf tables it is when I look at the /proc/modules file as described in the referenced article above (at least I see some entries there beginning with nftables... .

However in this case as my server is brand new with not much installed I would consider to switch to ubuntu as OS becasue Strato offers that also even as default setup.

Or does somebody know if there is another proxy manager that offers the IP banning feature out of the box - maybe does Traeffik do this?

You can use Traefik in combination with Crowdsec bouncers. I've heard those support iptables and nftables.

Otherwise, if you are behind Cloudflare, you may just use the Cloudflare API to ban threat actors there directly.

[l4rm4nd]

Though debian does not explicitely say that its nf tables

They do, see here.

"NOTE: iptables was replaced by nftables starting in Debian 10 Buster"

You may switch back to legacy iptables following these links:

• https://www.danami.com/clients/knowledgebase/194/How-can-I-switch-back-to-iptables-from-nftables-after-upgrading-to-Ubuntu.html
• https://wiki.debian.org/nftables#Reverting_to_legacy_xtables