search

MichaIng / DietPi

Lightweight justice for your single-board computer!
https://dietpi.com/
GNU General Public License v2.0
4.81k stars 494 forks source link

fail2ban not banning IP adresses and not noticing failed logins over SSH #3697

Closed Darwolia closed 4 years ago

Darwolia commented 4 years ago

Creating a bug report/issue

Required Information

Additional Information (if applicable)

Steps to reproduce

  1. Install fail2ban with the dietpi-software command
  2. Deliberately fail the login with PuTTY (with root user and an extra added user - with a RSA key-pair)

Expected behaviour

Actual behaviour

Extra details

Installed software since fresh install:

MichaIng commented 4 years ago

Many thanks for your report.

How exactly did you trigger the failed login? In case of a key pair, if you e.g. use a passphrase and enter it wrong, this is a pre-authentication failure on the client only and the server will not recognise about it at all.

Please check journalctl -u sshd if you see any logs about the failed login attempts.

Darwolia commented 4 years ago

Hello!

I did not know about entering a wrong passphrase beeing a failure on the clients side (I am using a passphrase)

I did some testing with root, an existing user, non existing user, a correct, wrong and no key-file (both with a passphrase) for login. In the sshd_config file I prohibited root login and password authentication.

Results:

User                root                  existing                not existing right key         not noticed      works                   noticed wrong key      not noticed      not noticed          noticed no key            not noticed      not noticed          noticed

Please excuse the formatting.

I have entered the correct passphrase for each key everytime. Guess my lack of knowledge was the issue and this behaviour is intented? Executing journalctl -u sshd afterwards shows no entries though. But I am not sure if that is a reason to keep this issue open.

-- Logs begin at Thu 2019-02-14 11:11:58 CET, end at Sat 2020-07-25 20:57:45 CEST. -- -- No entries --

MichaIng commented 4 years ago

Ah it should be journalctl -u ssh as the service name is ssh.service. Hmm probably the default sshd jail lacks a pattern for failed key authentication. When you enter the right key, it should produce an authentication error log and hence a fail2ban trigger.

Darwolia commented 4 years ago

For each failed login there are two lines generated: error: Received disconnect from IP address port 58060:14: No supported authentication methods available [pre auth] and Disconnected from authenticating user root IP address port 58059 [preauth]

The lines are always the same for root (prohibited by the sshd_config file) and my actual user, wether I use the right key file, the wrong one or none at all.

The only time it looks different is when I use a non existing username:

error: Received disconnect from IP address port 58067:14: No supported authentication methods available [pre auth] and Disconnected from invalid user asd IP address port 58067 [preauth]

MichaIng commented 4 years ago

This is how bots try to break in our server minutely:

Jul 26 11:20:11 dietpi.com sshd[2570408]: pam_unix(sshd:auth): check pass; user unknown
Jul 26 11:20:11 dietpi.com sshd[2570408]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=XXX.YY.Z.UUU
Jul 26 11:20:15 dietpi.com sshd[2570408]: error: maximum authentication attempts exceeded for invalid user admin from XXX.YY.Z.UUU port 34347 ssh2 [preauth]

The second error message from PAM and the one from the sshd outside of PAM (we allow only a single authentication attempt) are both tracked by fail2ban, hence count as two "attempts" which we must consider when setting the allowed retries before adding a ban.

If the user does exist, the first log line is missing, the second has user=<username> appended and third has invalid user replaced by the actual user name, which matches your case.

When I use a wrong key, I only see the last log (either with invalid user or with user name) which basically makes sense. Since the key authentication is done within the SSH server only, nothing is directed to PAM until the key authentication succeeded. Password authentication instead is done against the UNIX user password, hence is forwarded to PAM directly. Only after key authentication succeeded, it is forwarded to PAM to apply OS-side checks, which gives me then pam_unix(sshd:session): session opened for user <name> by (uid=X).

The PAM failure is caught by /etc/fail2ban/filter.d/sshd.conf regex:

^pam_unix\(sshd:auth\):\s+authentication failure;\s*logname=\S*\s*uid=\d*\s*euid=\d*\s*tty=\S*\s*ruser=<F-USER>\S*</F-USER>\s*rhost=<HOST>\s.*%(__suff)s$

The sshd error I get is covered by regex:

^(error: )?maximum authentication attempts exceeded for <F-USER>.*</F-USER> from <HOST>%(__on_port_opt)s(?: ssh\d*)?%(__suff)s$

We seem to always get the same sshd error which obviously overrides all other error messages that should be printed instead, although probably it is different when I prohibit password authentication for all accounts instead of just root. Regardless there is at least one error log with remove IP that is tracked and in case banned by fail2ban.

And now I see the reason why in your case it does not work. There is the regex:

^Received <F-MLFFORGET>disconnect</F-MLFFORGET> from <HOST>%(__on_port_opt)s:\s*14: No supported authentication methods available%(__suff)s$

But its part of the mdre-extra block and while I am no expert at those fail2ban filter, it looks quite clear that for this to be in effect one needs to set mode = extra. Please open /etc/fail2ban/fail.conf and below [sshd] add mode = extra:

[sshd]
mode = extra

Hmm not sure if this mode makes sense by default. I guess it is seen as too harmless when someone does a single authentication attempt with no or wrong key while key authentication is strictly required, not sure.

Btw [pre auth] is actually [preauth] with a line break inside, right? Because the first one would not be covered by any regex %(__suff)s 😉.

Darwolia commented 4 years ago

Yes, I copied the line breaks by accident. Adding mode = extra or even mode = aggressive beneath [sshd] in the /etc/fail2ban/jail.conf does not change the behaviour for some reason. But it seems to be a problem with fail2ban itself and not DietPi?

MichaIng commented 4 years ago

But it seems to be a problem with fail2ban itself and not DietPi?

Found the issue: Our jail.conf does not pass the mode variable: https://github.com/MichaIng/DietPi/blob/dev/dietpi/dietpi-software Changelog: https://github.com/MichaIng/DietPi/commit/29aedfef4933f641d44d2d772b26f71ecf64a2e0

Since most users will not know about this anyway, I added a brief info to our default config as well. What we probably could do is checking where the filter is coming from (upstream or Debian package maintainer) and suggesting to move the rule that affects your login failures to the "normal" mode. I mean the change that someone breaks a key by just trying is practically zero, but it is also about the overhead for the OpenSSH server. When the originating IP is blocked via blackhole route, there is nearly zero system load produced by connection attempts. And compared to the large amount of regex rules, one more should really not play a role for fail2ban log parsing on the other hand.

EDIT: Upstream filter: https://github.com/fail2ban/fail2ban/blob/master/config/filter.d/sshd.conf

Darwolia commented 4 years ago

Alright, with the changes to the jail.conf and sshd.conf preauth errors are noticed with mode = extra and the IP address is eventually banned. I'll close this issue, thanks for the help!