firewall ruleset with sequential port list results in blocked port

Zempashi commented 4 months ago

Bug Report

Compilation of ruleset is not happening strictly identical as nft tools does it, resulting in broken programmed rules in the kernel.

Description

In our evaluation setup we use a bastion to carry all talosctl command which is in the subnet of the controle-plane nodes. We end up with a firewall rule like this:

apiVersion: v1alpha1
kind: NetworkDefaultActionConfig
ingress: block
---     
apiVersion: v1alpha1
kind: NetworkRuleConfig         
name: node
portSelector:
  ports:
    - 50000 
    - 50001
    - 10250
    - 4240 # cilium                                             
  protocol: tcp                   
ingress:  
  - subnet: <controlplane_subnet>

Applying this ruleset results in blocking the port 50001 (but port 50000, 4240 and 10250 and other firewall rules are applied correctly and working correctly)

Logs

There nothing suspicious looking at the generated ruleset :

table inet talos {                                                                                                                    
        chain ingress {                                                                                                               
                [...]                   
                ip saddr <control_plane_subnet> tcp dport { 4240, 10250, 50000, 50001 } counter packets 25 bytes 1500 accept                  
                [...]
        }                                                                                                                             
}

After calling for help the nftables community (thanks to them btw), looks like the problem comes from rule compilation

$ nft -d netlink list ruleset
[...]
inet talos @__set5
    element 000052c3  : 1 [end]
    element 000051c3  : 0 [end]
    element 000051c3  : 1 [end]
    element 000050c3  : 0 [end]
    element 00000b28  : 1 [end]
    element 00000a28  : 0 [end]
    element 00009110  : 1 [end]
    element 00009010  : 0 [end]
[...]

if we apply ruleset via nft tools

nft list ruleset > rule.set
nft flush ruleset
nft -f rule.set

Now the port 50001 is opened correctly, we can diff the compiled set of ports

inet talos @__set5
    element 000052c3  : 1 [end]
    element 000050c3  : 0 [end]
    element 00000b28  : 1 [end]
    element 00000a28  : 0 [end]
    element 00009110  : 1 [end]
    element 00009010  : 0 [end]

So nft create a config that look like one generated by a range 50000-50001 (and this is the workaround) even with a port list ruleset

table inet talos {                                                                                                                    
        chain ingress {                                                                                                               
                [...]                   
                ip saddr <control_plane_subnet> tcp dport { 4240, 10250, 50000, 50001 } counter packets 38 bytes 2280 accept                  
                [...]
        }                                                                                                                             
}

This looks like a similar reproduction of:

https://github.com/google/nftables/issues/247

You may find interesting the discussion linked at the end of the issue:

https://github.com/ovn-org/ovn-kubernetes/pull/3709

Environment

Talos version: [talosctl version --nodes <problematic nodes>]

Client:                             
        Tag:         v1.7.1         
        SHA:         undefined      
        Built:                      
        Go version:  go1.22.3       
        OS/Arch:     linux/amd64    
Server:                             
        NODE:        10.19.220.1    
        Tag:         v1.7.4         
        SHA:         cb3a8308       
        Built:                      
        Go version:  go1.22.3       
        OS/Arch:     linux/amd64    
        Enabled:     RBAC

Kubernetes version: [kubectl version --short]

Client Version: v1.30.1                                
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.30.1

Platform:

AMD64 baremetal (metal iso)

smira commented 4 months ago

Thank you for reporting this.

smira commented 4 months ago

I believe the key problem here is the adjacent ports not being merged correctly when building an interval set, which leads to the match failure.

siderolabs / talos