/etc/fail2ban/ip.blacklist not getting banned IPs

[lls-hcr]

Hi, I have followed the instruction carefully but I have no IPs in /etc/fail2ban/ip.blacklist

I am running: Fail2Ban v0.10.2 on: Debian/Raspbian GNU/Linux 10 (buster)

The log doesn't show anything special. I can find after restarting fail2ban:

[...] NOTICE [blacklist] Flush ticket(s) with blacklist [...] INFO Jail 'blacklist' started [...] INFO [blacklist] Found 116.110.253.171 - 2019-11-19 17:41:19

I am not sure what additional info I can provide. Why would ip.blacklist no record banned IPs?

Thank you

[lls-hcr]

Can someone help?

I think that nothing happens to the ip.blacklist file upon restart. IPs that I have added manually are not sorted out, duplicates are not deleted and no IPs are added to iptable.

I have tested the commands below:

sudo sort -u /etc/fail2ban/ip.blacklist -o /etc/fail2ban/ip.blacklist

It worked fine, IPs are sorted. I then tried:

sudo cat /etc/fail2ban/ip.blacklist | while read IP; do iptables -I f2b-<name> 1 -s $IP -j DROP; done

That one returns:

-bash: name: No such file or directory
-bash: name: No such file or directory
-bash: name: No such file or directory

While sudo iptables -S returns:

[...]
-A f2b-apache-404 -s 41.63.167.76/32 -j REJECT --reject-with icmp-port-unreachable
-A f2b-apache-404 -s 132.232.51.143/32 -j REJECT --reject-with icmp-port-unreachable
-A f2b-apache-404 -s 123.206.226.149/32 -j REJECT --reject-with icmp-port-unreachable
-A f2b-apache-404 -s 106.13.44.54/32 -j REJECT --reject-with icmp-port-unreachable
-A f2b-apache-404 -j RETURN
[...]

Looking forward for any help. Thanks

[lls-hcr]

Nobody's home? I couldn't make it work.

[mitchellkrogza]

Post your jail settings and other config files.

[lls-hcr]

Here are the conf + jail.local files that I am using

f2b_files.zip

[mitchellkrogza]

It worked fine, IPs are sorted. I then tried:

sudo cat /etc/fail2ban/ip.blacklist | while read IP; do iptables -I f2b-<name> 1 -s $IP -j DROP; done

What is returned with just cat /etc/fail2ban/ip.blacklist

[lls-hcr]

This will return the one IP I have manually added into the ip.blaklist file

103.133.108.33

EDIT:

but if I add more IPs, the sorting command does the job

[mitchellkrogza]

Ok why then did previous attempt to do that return -bash: name: No such file or directory

[lls-hcr]

I did a few tests before with some IPs entered manually. Now, with only one IP left in the ip.blacklist file this is the output:

cat /etc/fail2ban/ip.blacklist

103.133.108.33

sudo cat /etc/fail2ban/ip.blacklist | while read IP; do iptables -I f2b-<name> 1 -s $IP -j DROP; done

-bash: name: No such file or directory

[mitchellkrogza]

Did you download the action.d and filter.d as raw files? Please try downloading both files again to make 100% sure there is no formatting errors of any sort. Also check who is owner of both of those files once downloaded and make sure it matches the permissions of all other fail2ban actions and filters.

so

wget https://raw.githubusercontent.com/mitchellkrogza/Fail2Ban-Blacklist-JAIL-for-Repeat-Offenders-with-Perma-Extended-Banning/master/action.d/blacklist.conf -O /etc/fail2ban/action.d/blacklist.conf

and

wget https://raw.githubusercontent.com/mitchellkrogza/Fail2Ban-Blacklist-JAIL-for-Repeat-Offenders-with-Perma-Extended-Banning/master/filter.d/blacklist.conf -O /etc/fail2ban/filter.d/blacklist.conf

Also delete your current ip.blacklist file and re-create it and set permissions on it

sudo rm /etc/fail2ban/ip.blacklist && sudo touch /etc/fail2ban/ip.blacklist

[lls-hcr]

Thank you for taking the time.

I downloaded before the raw files from the same commands above. I did it again just now with the codes you provided and have set permission to the new ip.blacklist I have re-created with sudo chmod 755 /etc/fail2ban/ip.blacklist

I have also restarted fail2ban

ls -l /etc/fail2ban/filter.d/blacklist.conf

-rw-r--r-- 1 root root 2076 Dec 1 11:05 /etc/fail2ban/filter.d/blacklist.conf

ls -l /etc/fail2ban/filter.d/sshd.conf

-rw-r--r-- 1 root root 5318 Jan 18 2018 /etc/fail2ban/filter.d/sshd.conf

Maybe I am doing something wrong... however, if I restart fail2ban, I don't see any change in the ip.blacklist file. The modification date/time is the one from the creation of the file.

[lls-hcr]

I also see an error in the fail2ban log, linked to /var/log/fail2ban.log.2.gz:

2019-12-01 11:10:18,021 fail2ban.jail           [12567]: INFO    Creating new jail 'blacklist' 
2019-12-01 11:10:18,022 fail2ban.jail           [12567]: INFO    Jail 'blacklist' uses pyinotify {} 
2019-12-01 11:10:18,029 fail2ban.jail           [12567]: INFO    Initiated 'pyinotify' backend 
2019-12-01 11:10:18,039 fail2ban.server         [12567]: INFO    Jail blacklist is not a JournalFilter instance 
2019-12-01 11:10:18,040 fail2ban.filter         [12567]: INFO    Added logfile: '/var/log/fail2ban.log' (pos = 89515, hash = f9940e0e604cda2ddf47f2d17f50b2ce740480e5) 
2019-12-01 11:10:18,043 fail2ban.filter         [12567]: INFO    Added logfile: '/var/log/fail2ban.log.1' (pos = 1792740, hash = e057a57e0d8f02835ca548e28774518d0d9305cc) 
2019-12-01 11:10:18,045 fail2ban.filter         [12567]: INFO    Added logfile: '/var/log/fail2ban.log.2.gz' (pos = 1597, hash = 9771c128db97c2144c40c7cb433a33988052a627) 
2019-12-01 11:10:18,045 fail2ban.filter         [12567]: WARNING Error decoding line from '/var/log/fail2ban.log.2.gz' with 'UTF-8'. Consider setting logencoding=utf-8 (or another appropria
te encoding) for this jail. Continuing to process line ignoring invalid characters: b'\xc2r\x8d\xc6\x00K\xccB\x1e\xbb\x8ep:Q\xcd\xc2\xc27u\x05og\x16\xf0\xa4\x8d\xcf\x0c\x80$m\xf2\xb8Y\x00\x
1c\xc9\xcarfa\x01\x15\x12\xf8\'\x9b\x05\xe0\x11\xd6l\xc4,\xb6\x81\xe6\x04\x059\xf2\x17\x84\xaf\x0b\xcdb}\xdc\xdf\xc0,\x06\xc2\xed\xf9\x83\xdc\x90\xe0\xa9[\x9eq7\xbd!\xd9\x87\xa5\x1d"E\xf6L\
x0b\xa3\xe0> \xc05\x8dy\x00 \x92\xb7\xe4\xd8\xf3\xfa\x98{\x7f\xbd\x7f~\xfb\xe5\xe9\xdd?\xe5\x05\xefO?\xfd\xf4\xcf?]\x8c\rl\xec\xb7\xaf\x90%\xfa\x8f\'\xc8\x12\x1d\x0f\x9f\xce\x0f\xf7\xd6\xe6
\x0e/\x9f\x0f\xbf\xbf\xbeX{\x7fxy\xfe\xfc\xf4\x97__\xef\xe1\xf9\xec\xe7~\x86T!d!\xec\x86\xf0x\xb0K\xf9\xe0\xbf\xc5\x05,\xfapb\x9dj\xa2\xe4\x00\xde \xa9\xb2\xcf_y\xd0\xba\xb8$\x8e]\x1d\xdd\x
c1\xe1\xf9\x0e\xc35\xe3\x99\xc6\xb7\x7f\xabw\xd5q\x02\xf2\x0f\xba\xcbzb\xe8\x84\x90W"\xef\x89\xa0c)\xf4\x05=\x10\xa8\xfct~{\x7fy=;\x916*\xcas\x0cj\xa8eH\x8b\xf3\x1c\x81\xee\n' 

[mitchellkrogza]

gzipped log files WILL create errors. Your must set your logrotate for those logs to not be compressed.

[lls-hcr]

Now, with 2 IPs added manually in the ip.blaklist (duplicates):

cat /etc/fail2ban/ip.blacklist

171.235.61.38 171.235.61.38

sudo sort -u /etc/fail2ban/ip.blacklist -o /etc/fail2ban/ip.blacklist

One duplicate is deleted

cat /etc/fail2ban/ip.blacklist

171.235.61.38

sudo cat /etc/fail2ban/ip.blacklist | while read IP; do iptables -I f2b-<name> 1 -s $IP -j DROP; done

-bash: name: No such file or directory

[mitchellkrogza]

Or change your jail setting as follows until you can remedy the gzipped log files

[blacklist]
enabled = true
logpath  = /var/log/fail2ban.log
filter = blacklist
banaction = blacklist
findtime = 31536000 ; 1 year
bantime = 31536000 ; 1 year
maxretry = 5

[mitchellkrogza]

Then do the following to diagnose any fail2ban startup errors.

sudo service fail2ban stop
sudo fail2ban-client -vvv -x stop
sudo fail2ban-client -vvv -x start

This will give you clear errors during startup

Once finished you simply use

sudo fail2ban-client -vvv -x stop
sudo service fail2ban restart

to get fail2ban running again

[lls-hcr]

Thanks, I will change the jail to avoid the gzipped log file error. But still no luck with the ip.blacklist

The output of the -vvv -x start command is long, but nothing looks strange. At the end I get:

[...]
 + 2546 B6F56AD0 fail2ban                  HEAVY CMD: ['start', 'blacklist']
 + 2566 B6F56AD0 fail2ban                  HEAVY OK : None
 + 2566 B6F56AD0 fail2ban                  HEAVY CMD: ['echo', 'Server ready']
 + 2578 B6F56AD0 fail2ban                  HEAVY OK : ['Server ready']
 + 2578 B6F56AD0 fail2ban.beautifier       HEAVY Beautify ['Server ready'] with ['echo', 'Server ready']
Server ready

[mitchellkrogza]

Try resetting permissions on the file

sudo chown root:root /etc/fail2ban/ip.blacklist
sudo chmod 755 /etc/fail2ban/ip.blacklist

[lls-hcr]

Thank you for all the advices. Unfortunately, still seems to have no action on the ip.blacklist.

sudo cat /etc/fail2ban/ip.blacklist | while read IP; do iptables -I f2b-<name> 1 -s $IP -j DROP; done

will return:

-bash: name: No such file or directory

[mitchellkrogza]

The name tag is triggered by the jail name [blacklist]

[lls-hcr]

Interestingly, now with:

/etc/fail2ban/jail.local

action_ = blacklist[name=%(__name__)s, bantime="%(bantime)s", port="%(port)s", protocol="%(protocol)s", chain="%(chain)s"]

Note: I have just replaced %(banaction)s[name=%(__name__)s with blacklist[name=%(__name__)s in the chain after action_ =.

Now, the ip.blacklist file is receiving IPs, and they are added in the iptables rules with DROP but they are also duplicated:

-A f2b-apache-noscript -s 183.56.211.150/32 -j DROP
-A f2b-apache-noscript -s 89.22.166.70/32 -j DROP
-A f2b-apache-noscript -s 82.56.62.181/32 -j DROP
-A f2b-apache-noscript -s 27.69.242.187/32 -j DROP
-A f2b-apache-noscript -s 183.56.211.150/32 -j DROP
-A f2b-apache-noscript -s 171.251.22.179/32 -j DROP
-A f2b-apache-noscript -s 171.235.61.38/32 -j DROP
-A f2b-apache-noscript -s 129.21.67.167/32 -j DROP
-A f2b-apache-noscript -j RETURN
-A f2b-apache-404 -s 183.56.211.150/32 -j DROP
-A f2b-apache-404 -s 89.22.166.70/32 -j DROP
-A f2b-apache-404 -s 82.56.62.181/32 -j DROP
-A f2b-apache-404 -s 27.69.242.187/32 -j DROP
-A f2b-apache-404 -s 183.56.211.150/32 -j DROP
-A f2b-apache-404 -s 171.251.22.179/32 -j DROP
-A f2b-apache-404 -s 171.235.61.38/32 -j DROP
-A f2b-apache-404 -s 129.21.67.167/32 -j DROP
-A f2b-apache-404 -j RETURN
-A f2b-sshd -s 89.22.166.70/32 -j DROP
-A f2b-sshd -s 82.56.62.181/32 -j DROP
-A f2b-sshd -s 27.69.242.187/32 -j DROP
-A f2b-sshd -s 171.251.22.179/32 -j DROP
-A f2b-sshd -s 171.235.61.38/32 -j DROP
-A f2b-sshd -s 129.21.67.167/32 -j DROP
-A f2b-sshd -s 89.22.166.70/32 -j DROP
-A f2b-sshd -s 82.56.62.181/32 -j DROP
-A f2b-sshd -s 27.69.242.187/32 -j DROP
-A f2b-sshd -s 183.56.211.150/32 -j DROP
-A f2b-sshd -s 171.251.22.179/32 -j DROP
-A f2b-sshd -s 171.235.61.38/32 -j DROP
-A f2b-sshd -s 129.21.67.167/32 -j DROP
-A f2b-sshd -j RETURN

Not sure things are correct..

[mitchellkrogza]

the IP's should be added under the filtername f2b-blacklist

-A f2b-blacklist -s x.x.x.x/32 -j DROP

[lls-hcr]

Yes, that is what I thought. I can replace [name=%(__name__)s by [name=blacklist but then it act strangely. I end up with multiple f2b-blacklist chains.

It seems that, for some reason, the tag %(__name__)s is not recognized when used with %(banaction)s[name=%(__name__)s. This is why I tried to replacing tags with the jail name.

[mitchellkrogza]

If I were you I would set all your jails to enabled = false set only blacklist to enabled = true and then diagnose. Trying to diagnose fail2ban issues with so many jails all active just makes tiresome work. Or just move your existing jail.local to jail.bak and then create a more lightweight version starting with blacklist only. Once that is fixed then bring your other jails back online. I have this action and filter running on many servers with different fail2ban versions too and have never seen this behaviour at all.

[lls-hcr]

That is indeed a good advice. I have completely removed f2b and reinstalled it. I reinstalled the blacklist.conf (filter and action) as well as the ip.blacklist files with the proper permissions. Unfortunately, the issue persists as there is still nothing happening in the ip.blacklist file. The log looks clean.

I don't think we are going to find a solution. Thanks for all the help.

2019-12-02 22:05:39,804 fail2ban.server         [3374]: INFO    Starting Fail2ban v0.10.2 
2019-12-02 22:05:39,811 fail2ban.database       [3374]: INFO    Connected to fail2ban persistent database '/var/lib/fail2ban/fail2ban.sqlite3' 
2019-12-02 22:05:39,814 fail2ban.jail           [3374]: INFO    Creating new jail 'sshd' 
2019-12-02 22:05:39,840 fail2ban.jail           [3374]: INFO    Jail 'sshd' uses pyinotify {} 
2019-12-02 22:05:39,848 fail2ban.jail           [3374]: INFO    Initiated 'pyinotify' backend 
2019-12-02 22:05:39,850 fail2ban.filter         [3374]: INFO      maxLines: 1 
2019-12-02 22:05:39,914 fail2ban.server         [3374]: INFO    Jail sshd is not a JournalFilter instance 
2019-12-02 22:05:39,915 fail2ban.filter         [3374]: INFO    Added logfile: '/var/log/auth.log' (pos = 302407, hash = d5d28ce2fafb19ce69d7ef86606c43e8cf6b41b1) 
2019-12-02 22:05:39,924 fail2ban.filter         [3374]: INFO      encoding: UTF-8 
2019-12-02 22:05:39,925 fail2ban.filter         [3374]: INFO      maxRetry: 2 
2019-12-02 22:05:39,925 fail2ban.filter         [3374]: INFO      findtime: 3600 
2019-12-02 22:05:39,926 fail2ban.actions        [3374]: INFO      banTime: 86400 
2019-12-02 22:05:39,931 fail2ban.jail           [3374]: INFO    Creating new jail 'blacklist' 
2019-12-02 22:05:39,931 fail2ban.jail           [3374]: INFO    Jail 'blacklist' uses pyinotify {} 
2019-12-02 22:05:39,939 fail2ban.jail           [3374]: INFO    Initiated 'pyinotify' backend 
2019-12-02 22:05:39,948 fail2ban.server         [3374]: INFO    Jail blacklist is not a JournalFilter instance 
2019-12-02 22:05:39,950 fail2ban.filter         [3374]: INFO    Added logfile: '/var/log/fail2ban.log' (pos = 20560, hash = 1f603e957ce4da64e201b00d22c6397eeff9395a) 
2019-12-02 22:05:39,951 fail2ban.filter         [3374]: INFO      encoding: UTF-8 
2019-12-02 22:05:39,952 fail2ban.filter         [3374]: INFO      maxRetry: 5 
2019-12-02 22:05:39,953 fail2ban.filter         [3374]: INFO      findtime: 31536000 
2019-12-02 22:05:39,954 fail2ban.actions        [3374]: INFO      banTime: 31536000 
2019-12-02 22:05:41,249 fail2ban.jail           [3374]: INFO    Jail 'sshd' started 
2019-12-02 22:05:41,263 fail2ban.jail           [3374]: INFO    Jail 'blacklist' started 
2019-12-02 22:05:41,459 fail2ban.actions        [3374]: NOTICE  [sshd] Restore Ban 121.142.165.111 

[lls-hcr]

Some news. The ip.blacklist file receives IPs if, in the DEFAULT section, I comment banaction = iptables-multiport and replace by banaction = blacklist

[DEFAULT]

banaction = blacklist
#banaction = iptables-multiport

However, the IPS are duplicated and IPs are not added to the iptables rules. Below is the output in ip.blacklist

45.95.168.105
67.84.240.215
45.95.168.105
115.159.122.71
67.84.240.215
180.76.57.182

Any ideas?

[lls-hcr]

This is very strange. I have now 2 jails: 1) sshd, and 2) Blacklist

The ip.blacklist file gets IPs when banaction = blacklist is the default and no banaction under [sshd]. If I add banaction = iptables-multiport under [sshd], then it does not work. Also, if I have banaction = iptables-multiport as default and banaction = blacklist under [blacklist] it does not work. It seems there is some kind of conflict with the two banaction commands.

[lls-hcr]

Sorry for multiplying messages!

The ip.blacklist receives IPs if all the banaction command are set as banaction = blacklist by default. That is both [sshd] and [blacklist] are set to the blacklist banaction.

Is that correct? Shouldn't the [sshd] be banaction = iptables-multiport?