crowdsecurity / cs-firewall-bouncer

Crowdsec bouncer written in golang for firewalls
MIT License
120 stars 43 forks source link

Problem with nftables firewall bouncer #182

Open PrOOnOOb opened 2 years ago

PrOOnOOb commented 2 years ago

Hello, i have a problem with the nftables bouncer there are many errors in the bouncer log files all errors

time="29-05-2022` 12:02:47" level=info msg="backend type : nftables"
time="29-05-2022 12:02:47" level=info msg="nftables initiated"
time="29-05-2022 12:02:47" level=info msg="Processing new and deleted decisions . . ."
time="29-05-2022 12:02:48" level=error msg="unable to commit add decisions Receive: netlink receive: no such file or directory"
time="29-05-2022 12:02:48" level=info msg="13641 decisions added"
time="29-05-2022 12:03:39" level=info msg="removing 'crowdsec' table"
time="29-05-2022 12:03:39" level=fatal msg="shutdown fail: Receive: netlink receive: no such file or directory"
time="29-05-2022 12:04:38" level=info msg="backend type : nftables"
time="29-05-2022 12:04:38" level=info msg="nftables initiated"
time="29-05-2022 12:04:38" level=info msg="Processing new and deleted decisions . . ."
time="29-05-2022 12:04:42" level=error msg="unable to commit add decisions Receive: netlink receive: no such file or directory"
time="29-05-2022 12:04:42" level=info msg="13641 decisions added"
time="29-05-2022 12:57:28" level=error msg="unable to commit delete decisions Receive: netlink receive: no such file or directory"

it seems like the bouncer can not create the needed tables if i create the tables manually, the tables stay empty.

table ip crowdsec {
}
table ip6 crowdsec {
}

Unless i restart the bouncer then the bouncer delete the tables but did not create new ones i don't use any firewall managing tools like ufw. Rules I created manually works fine

table inet filter {
    chain input {
        type filter hook input priority filter; policy accept;
        iif "lo" accept
        ip protocol icmp icmp type echo-request limit rate over 10/second burst 4 packets drop
        ip6 nexthdr ipv6-icmp icmpv6 type echo-request limit rate over 10/second burst 4 packets drop
        ct state established,related accept
        ip6 nexthdr ipv6-icmp icmpv6 type { destination-unreachable, packet-too-big, time-exceeded, parameter-problem, echo-request, mld-listener-query, mld-listener-report, mld-listener-done, nd-router-solicit, nd-router-advert, nd-neighbor-solicit, nd-neighbor-advert, ind-neighbor-solicit, ind-neighbor-advert, mld2-listener-report } accept
        ip protocol icmp icmp type { destination-unreachable, echo-request, router-advertisement, router-solicitation, time-exceeded, parameter-problem } accept
        ip protocol igmp accept
        tcp dport 5829 accept
        tcp dport 80 accept
        tcp dport 443 accept
        tcp dport 25 accept
        tcp dport 587 accept
        tcp dport 465 accept
        tcp dport 110 accept
        tcp dport 995 accept
        tcp dport 143 accept
        tcp dport 993 accept
        counter packets 17 bytes 771 drop
    }

    chain output {
        type filter hook output priority filter; policy accept;
    }

    chain forward {
        type filter hook forward priority filter; policy drop;
    }
}

i enabled the debugging mode for the bouncer the log is attached crowdsec-firewall-bouncer.log

it looks to me like according to the logs, the bouncer manages to create everything (table, chain, set, rule) The whole thing runs on a VServer with OpenVZ virtualization (Debian 11 Kernel 4.19.0 I have crowdsec on several other servers that are kvm virtualized, everything runs without problems. I have no way to test it on another OpenVZ server. I tried to repeat in an LXC container, everything works under the same conditions. Are problems with OpenVZ known? Or is it a different problem that I am overlooking?

sbs2001 commented 2 years ago

I can reproduce the issue by

  1. Keep the bouncer running.
  2. Delete ip table managed by the bouncer via sudo nft delete table ip crowdsec
  3. Create some decision sudo cscli decisions add --ip 1.2.3.4

The bouncer then emits the error

ERRO[27-07-2022 17:42:57] unable to commit add decisions Receive: netlink receive: no such file or directory 

Upon deleting some decision, bouncer emits the error

ERRO[27-07-2022 17:46:07] unable to commit delete decisions Receive: netlink receive: no such file or directory

@PrOOnOOb any chance there's some service deleting the table ?

PrOOnOOb commented 2 years ago

Hey no there is no service running that deletes rules or otherwise interferes with the firewall.