fail2ban module configuration: Fatal: can't open lock file /run/xtables.lock: Read-only file system

coretemp commented 6 years ago

Issue description

The fail2ban service returns:

.fail2ban-serve[610]: fail2ban.utils          [610]: ERROR   7fe5847617b0 -- stderr: "Fatal: can't open lock file /run/xtables.lock: Read-only file system"

Steps to reproduce

Open up port 22 to the Internet and wait for someone to attack your machine or just simulate an attack yourself.

  services.fail2ban.enable = true;

Expected behaviour:

No fatal messages.

Technical details

Stable branch, but likely in all NixOS versions.

system: "x86_64-linux"
- host os: Linux 4.9.86, NixOS, 18.03.git.03ba84e (Impala)
- multi-user?: yes
- sandbox: no
- version: nix-env (Nix) 2.0.2
- nixpkgs: /nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs

emmanuelrosa commented 6 years ago

Under technical details please add the output of nix-info -m.

I also recommend enabling fail2ban debugging which you can do like this:

services.fail2ban.daemonConfig = ''
  [Definition]
  loglevel  = DEBUG
  logtarget = SYSLOG
  socket    = /run/fail2ban/fail2ban.sock
  pidfile   = /run/fail2ban/fail2ban.pid
'';

This is identical to the default configuration except it changes the logging level to debug. That will allow you to replicate the issue and post a verbose log.

As I understand it, to make fail2ban block SSH logins for a given IP all you need are 5 failed login attempts. If that's the case, you can state that to make your "steps to reproduce" more precise.

coretemp commented 6 years ago

Since I improved security fail2ban will likely never generate another message, but I added the details you requested.

I will also deploy again with the changes you specified.

aanderse commented 5 years ago

@coretemp any update on this? I haven't had any issues with failban on 18.09.

CRTified commented 4 years ago

I stumbled across this problem and found the (suspected) culprit in my case.

As I disabled the firewall, fail2ban was not able to use iptables (which is used in the configuration for the default SSH jail, see here).

{
  users.users.root.password = "test";

  networking.firewall.enable = false;

  services = {
    fail2ban.enable = true;
    openssh.enable = true;
  };
}

Save it as mwe.nix and run this to test it in a VM

$(nix-build '<nixpkgs/nixos>' -A vm -I nixos-config=./mwe.nix)/bin/run-nixos-vm

When failing to login over SSH (use the non-loopback IP in the VM, as localhost is whitelisted by default), the error messages presented by @coretemp (and some more) appears in the journal and the IP is not banned (as not firewall is present to enforce the ban). Enabling the firewall results in everything working as expected.

It might make sense to issue a warning (or assertion) when fail2ban is enabled, but the firewall is disabled.

aanderse commented 4 years ago

@CRTified is that something you would follow up with in a PR and we could close this issue?

CRTified commented 4 years ago

@aanderse I would be able to do that, but I'd need some input on how to solve it:

Just add an assertion that services.fail2ban.enable == networking.firewall.enable
Remove the default jails, as it is possible to use fail2ban without iptables (e.g. to send mails or run custom commands)

I prefer the first approach, as the available actions/filters are hardcoded and this is probably the easiest solution.

But as it might be useful to lift that restriction (so that it's possible to add custom filters like a whitelisting approach), the second approach might be more useful (although adding custom filters is probably used only occasionally).

aanderse commented 4 years ago

cc @Izorkin @Mic92 @flokli @Infinisil

Izorkin commented 4 years ago

Just add an assertion that services.fail2ban.enable == networking.firewall.enable

no ideas how to implement

Remove the default jails, as it is possible to use fail2ban without iptables (e.g. to send mails or run custom commands)

@CRTified not checked new variant fail2ban service? - #67931

CRTified commented 4 years ago

no ideas how to implement

Basically by adding something like this:

assertions = [
  {
    assertion = config.services.fail2ban.enable == config.networking.firewall.enable;
    message = "fail2ban can not be used without a firewall";
  }
];

This issue is the result of being able to start fail2ban with the default actions without having a firewall available. I did not notice the new variant, but as it allows using something else than iptables, that assertion is not useful.

I would suggest that the new version for the fail2ban service resolves this issue, as it gives direct access to the used ban action (and allows custom actions/filters by using a different/custom package). So it should hopefully be more or less clear that the ban action needs to be available (and it is probably impossible to check whether the ban action works when building the system).

stale[bot] commented 4 years ago

Hello, I'm a bot and I thank you in the name of the community for opening this issue.

To help our human contributors focus on the most-relevant reports, I check up on old issues to see if they're still relevant. This issue has had no activity for 180 days, and so I marked it as stale, but you can rest assured it will never be closed by a non-human.

The community would appreciate your effort in checking if the issue is still valid. If it isn't, please close it.

If the issue persists, and you'd like to remove the stale label, you simply need to leave a comment. Your comment can be as simple as "still important to me". If you'd like it to get more attention, you can ask for help by searching for maintainers and people that previously touched related code and @ mention them in a comment. You can use Git blame or GitHub's web interface on the relevant files to find them.

Lastly, you can always ask for help at our Discourse Forum or at #nixos' IRC channel.

NixOS / nixpkgs