Document how nftables and firewalld direct work together

[erinn]

I have read this: https://firewalld.org/2018/07/nftables-backend

The pertinent sections seem to be:

What about the direct interface? Seasoned firewalld users may already be asking themselves, What about the direct interface? No worries, it’s still there and works almost exactly as it did in previous releases. When the nftables backend is enabled direct rules are treated specially and still use iptables and family. This means your existing configurations with custom direct rules will continue to work. However, there is one deliberate behavioral change - direct rules take precedence over all other firewalld rules. For further details see the Behavorial Changes section below.

Fine, direct rules will work. Everything continues on as normal.

Direct interface precedence In previous versions of firewalld rules added via the direct interface were still subject to firewalld’s rules that occur early in the ruleset. Most notably is the early acceptance of packets that are part of existing connections. This has led to confusion for users, for example see github issue 44.

With the nftables backend direct rules are deliberately given a higher precedence than all other firewalld rules. Now when a user adds a direct rule to drop traffic existing connections will also be affected.

Even better, direct rules will for sure work, excellent!

Packet accept/drop precedence As mentioned above, nftables allows multiple chains to use the same netfilter hook. A consequence of this is that packets that are accepted are still subject to the rules of other chains hooked into the hook type. For firewalld this means packets may be accepted early by custom iptables or nftables rules, but will still be subject to firewalld’s rules. In the drop case processing always stops immediately and no other hooks will process the packet.

Wait... direct rules won't work? I'm confused.

So I test on a RHEL 8.1 system:

[root@localhost ~]# firewall-cmd --get-active-zones
drop
  interfaces: eth0
[root@localhost ~]# firewall-cmd --direct --add-rule ipv4 filter INPUT 50 -p tcp -m tcp -m multiport --dports 22 -m comment --comment 'allow world to ssh' -j ACCEPT
[root@localhost ~]# iptables -L -n
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp multiport dports 22 /* allow world to ssh */

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination 

SSH should work right? Nope. Perhaps it is the drop, so I test:

[root@localhost ~]# firewall-cmd --set-default-zone=block
success
[root@localhost ~]# firewall-cmd --get-active-zones
block
  interfaces: eth0
[root@localhost ~]# iptables -L -n
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp multiport dports 22 /* allow world to ssh */

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination 

Nope, doesn't work.

I can make it work like so, but that is basically disabling the firewall: firewall-cmd --set-default-zone=trusted

So what is the workflow for using direct rules in firewalld with nftables as the back end? Is there one? The documentation seems to imply so, my reality seems to imply not. Further I don't see the link between nftables and iptables, perhaps it is implicit, but I keep looking for something in nft list ruleset that will show me a low precedence jump to the iptables rules, it appears not to exist. Perhaps it is hidden from users? Or perhaps I am looking in the wrong place?

Finally I understand that in my simple example firewalld can be made to do it just fine, but I am trying to figure out how to get the direct rules working. In my specific case the chef firewall cookbook chose to go the route of using direct rules for everything, and if direct rules don't work now... well there is gonna be a lot of work in the chef cookbook to get it working with firewalld and nftables.

Any explanation is much appreciated, and if possible please add it to the documentation, I suspect I am not the only one suffering under confusion from this issue.

Thanks, -Erinn

[erig0]

[..]

Packet accept/drop precedence As mentioned above, nftables allows multiple chains to use the same netfilter hook. A consequence of this is that packets that are accepted are still subject to the rules of other chains hooked into the hook type. For firewalld this means packets may be accepted early by custom iptables or nftables rules, but will still be subject to firewalld’s rules. In the drop case processing always stops immediately and no other hooks will process the packet.

Wait... direct rules won't work? I'm confused.

They still work, but the behavior is slightly different. That's what this whole section is about.

So I test on a RHEL 8.1 system:

[root@localhost ~]# firewall-cmd --get-active-zones
drop
  interfaces: eth0
[root@localhost ~]# firewall-cmd --direct --add-rule ipv4 filter INPUT 50 -p tcp -m tcp -m multiport --dports 22 -m comment --comment 'allow world to ssh' -j ACCEPT
[root@localhost ~]# iptables -L -n
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp multiport dports 22 /* allow world to ssh */

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination 

SSH should work right? Nope.

No. The traffic still has to get through the standard firewalld rules. The direct rules and standard firewalld rules are separate. The behavior you're seeing here is exactly what is described in the "Packet accept/drop precedence" above.

Perhaps it is the drop, so I test:

[root@localhost ~]# firewall-cmd --set-default-zone=block
success
[root@localhost ~]# firewall-cmd --get-active-zones
block
  interfaces: eth0
[root@localhost ~]# iptables -L -n
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp multiport dports 22 /* allow world to ssh */

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination 

Nope, doesn't work.

Right. It's still being blocked by firewalld.

I can make it work like so, but that is basically disabling the firewall: firewall-cmd --set-default-zone=trusted

Makes sense. You're allowing the traffic in firewalld. Although you should instead enable the ssh service in the appropriate zone.

So what is the workflow for using direct rules in firewalld with nftables as the back end? Is there one? The documentation seems to imply so, my reality seems to imply not. Further I don't see the link between nftables and iptables, perhaps it is implicit,

There is no user facing link between iptables an nftables. They are separate firewalls. In the kernel they do share some infrastructure though.

but I keep looking for something in nft list ruleset that will show me a low precedence jump to the iptables rules, it appears not to exist. Perhaps it is hidden from users? Or perhaps I am looking in the wrong place?

nftables does not do a "jump" to iptables. In standard setups iptables and nftables have the same precedence. However, firewalld explicitly orders its nftables rules after iptables (see "Direct interface precedence" above). This is so it's deterministic.

Finally I understand that in my simple example firewalld can be made to do it just fine, but I am trying to figure out how to get the direct rules working.

Filing this issue and asking for help is a good start. :)

In my specific case the chef firewall cookbook chose to go the route of using direct rules for everything, and if direct rules don't work now... well there is gonna be a lot of work in the chef cookbook to get it working with firewalld and nftables.

I don't get why you'd force firewalld only to use direct rules exclusively. Firewalld provides service, ports, rich rules, etc to make things easier. direct rules should be used as a last resort.

[erinn]

I can't speak to why chef chose to use direct rules for their firewall cookbook, I suspect it was just ease, but regardless that was they choice they made.

In my contrived example where I can only use direct rules, can you provide me an example of how it should work? I want to allow ssh from all deny everything else, because as mentioned, the only way I got it to work was to set the zone to trusted which allowed all. What's the right way?

Finally I understand that direct rules are strongly discouraged, should that particular interface perhaps be deprecated? I am just struggling with how it is useful at this point short of switching back to iptables as the back end?

Thanks for your time, -Erinn

[erig0]

I can't speak to why chef chose to use direct rules for their firewall cookbook, I suspect it was just ease, but regardless that was they choice they made.

In my contrived example where I can only use direct rules, can you provide me an example of how it should work?

If you can only use direct rules, then you'll have to switch the FirewallBackend to iptables.

I want to allow ssh from all deny everything else, because as mentioned, the only way I got it to work was to set the zone to trusted which allowed all. What's the right way?

FWIW, this is a very simple use case that should be done firewalld primitives (zones, services).

If you insist on using direct rules, and if you can use other firewalld features, then you should add ssh to the appropriate zone (it's likely already enabled in the default zone, public, so you may not have to do anything). e.g.

firewall-cmd --zone=foobar --add-service=ssh

Finally I understand that direct rules are strongly discouraged, should that particular interface perhaps be deprecated?

I would love to deprecate it, but there are still many things we can't do without direct rules, e.g. FORWARD/OUTPUT filtering.

I am just struggling with how it is useful at this point short of switching back to iptables as the back end?

If you use direct rules for dropping traffic, then there is no issue. All still works the same. This is the case for things like fail2ban. But it is trickier if you want to ACCEPT traffic via direct rules.

Can you post the direct rules you're creating via chef? It may help decide what's best in your scenario.

[erinn]

This has less to do with what we are doing, and more to do with how chef is doing it. But to give you the broad idea it is a simple deny everything and allow some things firewall configuration for incoming traffic. Chef however has chosen to implement this through direct rules in firewalld. You can see their code here: https://github.com/chef-cookbooks/firewall if you are interested. And the actual insertion rule here: https://github.com/chef-cookbooks/firewall/blob/master/libraries/provider_firewall_firewalld.rb#L68

I understand that our use case can be covered by firewalld without direct rules, I understand that it is very simple, but I needed to get a grasp on whether it was even possible to have direct allow rules. It appears that answer is no unless I switch the backend to iptables.

So at this point I'll be opening an issue with the chef folks about this for their firewall cookbook. Hopefully they can get it fixed, but for the moment, in a default configuration chef's firewall cookbook and firewalld don't appear to be working together.

Thanks, -Erinn

[lastmikoi]

Thank you @erinn for having filed this issue and saving me a couple hours of debugging, as I have run into the same issue, but with a different usercase.

Indeed, I was expecting to be able to ACCEPT traffic via direct rules, as I am trying to do non-NAT IP forwarding using firewalld with nftables (CentOS 8 router). However, since it's not possible to define FORWARD rules outside of direct rules and there is a default reject in filter_FORWARD, we end up with firewalld being, apparently, impossible to use in a non-masquerading EL8 router, unless ones reverts to using iptables as a backend.

I'm looking forward (pun intended :) ) to FORWARD/OUTPUT filtering being implemented, but in the mean time I am afraid I have to choose between using firewalld or using nftables.

[alexisfrjp]

I wish I had found this post earlier. Especially the blog post explaining the precedences.