QubesOS / qubes-issues

The Qubes OS Project issue tracker
https://www.qubes-os.org/doc/issue-tracking/
534 stars 47 forks source link

Integrate Whonix firewall with Qubes nftables rules #8562

Open adrelanos opened 1 year ago

adrelanos commented 1 year ago

Whonix firewall making progress towards nftables and IPv6 support (https://github.com/QubesOS/qubes-issues/issues/9267).

/usr/bin/whonix-gateway-firewall.nftables was historically scripting based. That worked well for iptables but nftables shells scripting is discouraged. [1]

(The .nftables suffix is only to ease testing. It will be dropped in the final version. Will replace the old iptables version /usr/bin/whonix-gateway-firewall.)

To keep the delta small, easier to review,

  1. the Whonix firewall bash script writes an nftables firewall script and stores it in file /var/lib/whonix-firewall/firewall.nft.
  2. executes that script.

This is a solution for the nftables shell scripting issue. Here is the work in progress version of that generated nftables script:

#!/usr/sbin/nft -f
flush ruleset
add table nat
create table ip filter
add chain filter INPUT { type filter hook input priority 0; policy drop; }
add chain filter FORWARD { type filter hook forward priority 0; policy drop; }
add chain filter OUTPUT { type filter hook output priority 0; policy drop; }
add chain nat PREROUTING { type nat hook prerouting priority -100; }
add chain nat OUTPUT { type nat hook output priority -100; }
add rule ip filter INPUT ct state invalid counter drop
add rule ip filter INPUT tcp flags & (fin|syn|rst|psh|ack|urg) == fin|syn|rst|ack counter drop
add rule ip filter INPUT tcp flags & (fin|syn) == fin|syn counter drop
add rule ip filter INPUT tcp flags & (syn|rst) == syn|rst counter drop
add rule ip filter INPUT ip frag-off & 0x1fff != 0 counter drop
add rule ip filter INPUT tcp flags & (fin|syn|rst|psh|ack|urg) == fin|syn|rst|psh|ack|urg counter drop
add rule ip filter INPUT tcp flags & (fin|syn|rst|psh|ack|urg) == 0x0 counter drop
add rule ip filter INPUT iifname vif* tcp dport 8082 counter accept
add rule ip filter OUTPUT oifname vif* tcp sport 8082 counter accept
add chain ip nat PR-QBS-SERVICES
add rule ip nat PREROUTING counter jump PR-QBS-SERVICES
add rule ip nat PR-QBS-SERVICES iifname vif* ip daddr 10.137.255.254 tcp dport 8082 counter redirect
add rule ip nat OUTPUT ip protocol udp skuid tinyproxy ct state new counter dnat to 127.0.0.1:5400
add rule ip nat OUTPUT ip protocol tcp skuid tinyproxy ct state new counter dnat to 127.0.0.1:9041
add rule ip filter OUTPUT ip daddr 127.0.0.1 skuid tinyproxy ct state new udp dport 5400 counter accept
add rule ip filter OUTPUT ip daddr 127.0.0.1 skuid tinyproxy ct state new tcp dport 9041 counter accept
add rule ip filter INPUT iifname lo counter accept
add rule ip filter INPUT ct state established counter accept
add rule filter INPUT icmp type destination-unreachable icmp code frag-needed ct state related counter accept
add rule ip filter INPUT ip protocol icmp counter drop
add rule ip filter INPUT iifname eth0 ip protocol tcp tcp dport { 22,9050,9051,9150,9151 } counter accept
add rule ip filter INPUT iifname eth0 ip protocol udp udp dport { 22,9050,9051,9150,9151 } counter accept
add rule ip filter INPUT iifname vif* udp dport 5300 counter accept
add rule ip filter INPUT iifname vif* tcp dport 9040 counter accept
add rule ip filter INPUT iifname vif* tcp dport 9051 counter accept
add rule ip filter INPUT iifname vif* tcp dport 9050 counter accept
add rule ip filter INPUT iifname vif* tcp dport 9100 counter accept
add rule ip filter INPUT iifname vif* tcp dport 9101 counter accept
add rule ip filter INPUT iifname vif* tcp dport 9102 counter accept
add rule ip filter INPUT iifname vif* tcp dport 9103 counter accept
add rule ip filter INPUT iifname vif* tcp dport 9104 counter accept
add rule ip filter INPUT iifname vif* tcp dport 9105 counter accept
add rule ip filter INPUT iifname vif* tcp dport 9106 counter accept
add rule ip filter INPUT iifname vif* tcp dport 9107 counter accept
add rule ip filter INPUT iifname vif* tcp dport 9108 counter accept
add rule ip filter INPUT iifname vif* tcp dport 9109 counter accept
add rule ip filter INPUT iifname vif* tcp dport 9110 counter accept
add rule ip filter INPUT iifname vif* tcp dport 9111 counter accept
add rule ip filter INPUT iifname vif* tcp dport 9114 counter accept
add rule ip filter INPUT iifname vif* tcp dport 9115 counter accept
add rule ip filter INPUT iifname vif* tcp dport 9117 counter accept
add rule ip filter INPUT iifname vif* tcp dport 9118 counter accept
add rule ip filter INPUT iifname vif* tcp dport 9122 counter accept
add rule ip filter INPUT iifname vif* tcp dport 9123 counter accept
add rule ip filter INPUT iifname vif* tcp dport 9124 counter accept
add rule ip filter INPUT iifname vif* tcp dport 9125 counter accept
add rule ip filter INPUT iifname vif* tcp dport 9150 counter accept
add rule ip filter INPUT iifname vif* tcp dport 9152-9189 counter accept
add rule ip nat PREROUTING iifname vif* ip daddr 10.137.0.0/16 tcp dport 9051 counter redirect to :9051
add rule ip nat PREROUTING iifname vif* ip daddr 10.138.0.0/16 tcp dport 9051 counter redirect to :9051
add rule ip nat PREROUTING iifname vif* ip daddr 10.152.152.10 tcp dport 9051 counter redirect to :9051
add rule ip nat PREROUTING iifname vif* ip daddr 10.137.0.0/16 tcp dport 9050 counter redirect to :9050
add rule ip nat PREROUTING iifname vif* ip daddr 10.138.0.0/16 tcp dport 9050 counter redirect to :9050
add rule ip nat PREROUTING iifname vif* ip daddr 10.152.152.10 tcp dport 9050 counter redirect to :9050
add rule ip nat PREROUTING iifname vif* ip daddr 10.137.0.0/16 tcp dport 9100 counter redirect to :9100
add rule ip nat PREROUTING iifname vif* ip daddr 10.138.0.0/16 tcp dport 9100 counter redirect to :9100
add rule ip nat PREROUTING iifname vif* ip daddr 10.152.152.10 tcp dport 9100 counter redirect to :9100
add rule ip nat PREROUTING iifname vif* ip daddr 10.137.0.0/16 tcp dport 9101 counter redirect to :9101
add rule ip nat PREROUTING iifname vif* ip daddr 10.138.0.0/16 tcp dport 9101 counter redirect to :9101
add rule ip nat PREROUTING iifname vif* ip daddr 10.152.152.10 tcp dport 9101 counter redirect to :9101
add rule ip nat PREROUTING iifname vif* ip daddr 10.137.0.0/16 tcp dport 9102 counter redirect to :9102
add rule ip nat PREROUTING iifname vif* ip daddr 10.138.0.0/16 tcp dport 9102 counter redirect to :9102
add rule ip nat PREROUTING iifname vif* ip daddr 10.152.152.10 tcp dport 9102 counter redirect to :9102
add rule ip nat PREROUTING iifname vif* ip daddr 10.137.0.0/16 tcp dport 9103 counter redirect to :9103
add rule ip nat PREROUTING iifname vif* ip daddr 10.138.0.0/16 tcp dport 9103 counter redirect to :9103
add rule ip nat PREROUTING iifname vif* ip daddr 10.152.152.10 tcp dport 9103 counter redirect to :9103
add rule ip nat PREROUTING iifname vif* ip daddr 10.137.0.0/16 tcp dport 9104 counter redirect to :9104
add rule ip nat PREROUTING iifname vif* ip daddr 10.138.0.0/16 tcp dport 9104 counter redirect to :9104
add rule ip nat PREROUTING iifname vif* ip daddr 10.152.152.10 tcp dport 9104 counter redirect to :9104
add rule ip nat PREROUTING iifname vif* ip daddr 10.137.0.0/16 tcp dport 9105 counter redirect to :9105
add rule ip nat PREROUTING iifname vif* ip daddr 10.138.0.0/16 tcp dport 9105 counter redirect to :9105
add rule ip nat PREROUTING iifname vif* ip daddr 10.152.152.10 tcp dport 9105 counter redirect to :9105
add rule ip nat PREROUTING iifname vif* ip daddr 10.137.0.0/16 tcp dport 9106 counter redirect to :9106
add rule ip nat PREROUTING iifname vif* ip daddr 10.138.0.0/16 tcp dport 9106 counter redirect to :9106
add rule ip nat PREROUTING iifname vif* ip daddr 10.152.152.10 tcp dport 9106 counter redirect to :9106
add rule ip nat PREROUTING iifname vif* ip daddr 10.137.0.0/16 tcp dport 9107 counter redirect to :9107
add rule ip nat PREROUTING iifname vif* ip daddr 10.138.0.0/16 tcp dport 9107 counter redirect to :9107
add rule ip nat PREROUTING iifname vif* ip daddr 10.152.152.10 tcp dport 9107 counter redirect to :9107
add rule ip nat PREROUTING iifname vif* ip daddr 10.137.0.0/16 tcp dport 9108 counter redirect to :9108
add rule ip nat PREROUTING iifname vif* ip daddr 10.138.0.0/16 tcp dport 9108 counter redirect to :9108
add rule ip nat PREROUTING iifname vif* ip daddr 10.152.152.10 tcp dport 9108 counter redirect to :9108
add rule ip nat PREROUTING iifname vif* ip daddr 10.137.0.0/16 tcp dport 9109 counter redirect to :9109
add rule ip nat PREROUTING iifname vif* ip daddr 10.138.0.0/16 tcp dport 9109 counter redirect to :9109
add rule ip nat PREROUTING iifname vif* ip daddr 10.152.152.10 tcp dport 9109 counter redirect to :9109
add rule ip nat PREROUTING iifname vif* ip daddr 10.137.0.0/16 tcp dport 9110 counter redirect to :9110
add rule ip nat PREROUTING iifname vif* ip daddr 10.138.0.0/16 tcp dport 9110 counter redirect to :9110
add rule ip nat PREROUTING iifname vif* ip daddr 10.152.152.10 tcp dport 9110 counter redirect to :9110
add rule ip nat PREROUTING iifname vif* ip daddr 10.137.0.0/16 tcp dport 9111 counter redirect to :9111
add rule ip nat PREROUTING iifname vif* ip daddr 10.138.0.0/16 tcp dport 9111 counter redirect to :9111
add rule ip nat PREROUTING iifname vif* ip daddr 10.152.152.10 tcp dport 9111 counter redirect to :9111
add rule ip nat PREROUTING iifname vif* ip daddr 10.137.0.0/16 tcp dport 9114 counter redirect to :9114
add rule ip nat PREROUTING iifname vif* ip daddr 10.138.0.0/16 tcp dport 9114 counter redirect to :9114
add rule ip nat PREROUTING iifname vif* ip daddr 10.152.152.10 tcp dport 9114 counter redirect to :9114
add rule ip nat PREROUTING iifname vif* ip daddr 10.137.0.0/16 tcp dport 9115 counter redirect to :9115
add rule ip nat PREROUTING iifname vif* ip daddr 10.138.0.0/16 tcp dport 9115 counter redirect to :9115
add rule ip nat PREROUTING iifname vif* ip daddr 10.152.152.10 tcp dport 9115 counter redirect to :9115
add rule ip nat PREROUTING iifname vif* ip daddr 10.137.0.0/16 tcp dport 9117 counter redirect to :9117
add rule ip nat PREROUTING iifname vif* ip daddr 10.138.0.0/16 tcp dport 9117 counter redirect to :9117
add rule ip nat PREROUTING iifname vif* ip daddr 10.152.152.10 tcp dport 9117 counter redirect to :9117
add rule ip nat PREROUTING iifname vif* ip daddr 10.137.0.0/16 tcp dport 9118 counter redirect to :9118
add rule ip nat PREROUTING iifname vif* ip daddr 10.138.0.0/16 tcp dport 9118 counter redirect to :9118
add rule ip nat PREROUTING iifname vif* ip daddr 10.152.152.10 tcp dport 9118 counter redirect to :9118
add rule ip nat PREROUTING iifname vif* ip daddr 10.137.0.0/16 tcp dport 9122 counter redirect to :9122
add rule ip nat PREROUTING iifname vif* ip daddr 10.138.0.0/16 tcp dport 9122 counter redirect to :9122
add rule ip nat PREROUTING iifname vif* ip daddr 10.152.152.10 tcp dport 9122 counter redirect to :9122
add rule ip nat PREROUTING iifname vif* ip daddr 10.137.0.0/16 tcp dport 9123 counter redirect to :9123
add rule ip nat PREROUTING iifname vif* ip daddr 10.138.0.0/16 tcp dport 9123 counter redirect to :9123
add rule ip nat PREROUTING iifname vif* ip daddr 10.152.152.10 tcp dport 9123 counter redirect to :9123
add rule ip nat PREROUTING iifname vif* ip daddr 10.137.0.0/16 tcp dport 9124 counter redirect to :9124
add rule ip nat PREROUTING iifname vif* ip daddr 10.138.0.0/16 tcp dport 9124 counter redirect to :9124
add rule ip nat PREROUTING iifname vif* ip daddr 10.152.152.10 tcp dport 9124 counter redirect to :9124
add rule ip nat PREROUTING iifname vif* ip daddr 10.137.0.0/16 tcp dport 9125 counter redirect to :9125
add rule ip nat PREROUTING iifname vif* ip daddr 10.138.0.0/16 tcp dport 9125 counter redirect to :9125
add rule ip nat PREROUTING iifname vif* ip daddr 10.152.152.10 tcp dport 9125 counter redirect to :9125
add rule ip nat PREROUTING iifname vif* ip daddr 10.137.0.0/16 tcp dport 9150 counter redirect to :9150
add rule ip nat PREROUTING iifname vif* ip daddr 10.138.0.0/16 tcp dport 9150 counter redirect to :9150
add rule ip nat PREROUTING iifname vif* ip daddr 10.152.152.10 tcp dport 9150 counter redirect to :9150
add rule ip nat PREROUTING iifname vif* ip daddr 10.137.0.0/16 tcp dport 9152-9189 counter redirect
add rule ip nat PREROUTING iifname vif* ip daddr 10.138.0.0/16 tcp dport 9152-9189 counter redirect
add rule ip nat PREROUTING iifname vif* ip daddr 10.152.152.10 tcp dport 9152-9189 counter redirect
add rule ip nat PREROUTING iifname vif* udp dport 53 counter redirect to :5300
add rule ip nat PREROUTING iifname vif* tcp flags & (fin|syn|rst|ack) == syn counter redirect to :9040
add rule ip filter INPUT counter drop
add rule ip filter FORWARD counter reject with icmp type admin-prohibited
add rule ip filter OUTPUT ct state invalid counter reject with icmp type admin-prohibited
add rule ip filter OUTPUT ct state invalid counter reject with icmp type admin-prohibited
add rule ip filter OUTPUT tcp flags & (fin|syn|rst|psh|ack|urg) == fin|syn|rst|ack counter reject with icmp type admin-prohibited
add rule ip filter OUTPUT tcp flags & (fin|syn) == fin|syn counter reject with icmp type admin-prohibited
add rule ip filter OUTPUT tcp flags & (syn|rst) == syn|rst counter reject with icmp type admin-prohibited
add rule ip filter OUTPUT ip frag-off & 0x1fff != 0 counter reject with icmp type admin-prohibited
add rule ip filter OUTPUT tcp flags & (fin|syn|rst|psh|ack|urg) == fin|syn|rst|psh|ack|urg counter reject with icmp type admin-prohibited
add rule ip filter OUTPUT tcp flags & (fin|syn|rst|psh|ack|urg) == 0x0 counter reject with icmp type admin-prohibited
add rule ip nat OUTPUT skuid 104 counter return
add rule ip nat OUTPUT skuid 102 counter return
add rule ip nat OUTPUT skuid 100 counter return
add rule ip filter OUTPUT ct state established counter accept
add rule ip filter OUTPUT oifname lo counter accept
add rule ip filter OUTPUT skuid 104 counter accept
add rule ip filter OUTPUT skuid 102 counter accept
add rule ip filter OUTPUT skuid 100 counter accept
add rule ip filter OUTPUT counter reject with icmp type admin-prohibited

(iptables commands have been rewritten to nftables. Previous nftables commands are persisted as script comments for now comparison purposes.)

The first line of the Whonix nftables script is:

flush ruleset

This seems to be good practice. And there's also no race condition due to nftables atomic replacement. However, this results in kicking out the Qubes nftables rules. This is what I mean by Qubes nftables rules that:

table ip qubes {
    set downstream {
        type ipv4_addr
        elements = { 10.137.0.68, 10.137.0.84 }
    }

    set allowed {
        type ifname . ipv4_addr
        elements = { "vif9.0" . 10.137.0.68,
                 "vif11.0" . 10.137.0.84 }
    }

    chain prerouting {
        type filter hook prerouting priority raw; policy accept;
        iifgroup 2 goto antispoof
        ip saddr @downstream counter drop
    }

    chain antispoof {
        iifname . ip saddr @allowed accept
        counter drop
    }
}
table ip6 qubes {
    set downstream {
        type ipv6_addr
    }

    set allowed {
        type ifname . ipv6_addr
    }

    chain antispoof {
        iifname . ip6 saddr @allowed accept
        counter drop
    }

    chain prerouting {
        type filter hook prerouting priority raw; policy accept;
        iifgroup 2 goto antispoof
        ip6 saddr @downstream counter drop
    }
}
table inet qubes-nat-accel {
    flowtable qubes-accel {
        hook ingress priority filter
        devices = { eth0, eth1, lo, vif11.0, vif9.0 }
    }

    chain qubes-accel {
        type filter hook forward priority filter + 5; policy accept;
        meta l4proto { tcp, udp } iifgroup 2 oifgroup 1 flow add @qubes-accel
        counter
    }
}

How do you suggest /usr/bin/whonix-gateway-firewall.nftables (.nftabels) should load these Qubes nftables rules? Append the Qubes nftables rules source files to /var/lib/whonix-firewall/firewall.nft, run some systemd unit or script?


[1] https://wiki.nftables.org/wiki-nftables/index.php/Atomic_rule_replacement#Warning_about_Shell_scripting_.2B_nftables

DemiMarie commented 11 months ago

nftables supports file inclusion IIRC

adrelanos commented 11 months ago

Yes. Once I know which nftables rules to include I can certainly append them in the right place.

Which nftables rules (files) should I include?

https://github.com/QubesOS/qubes-core-agent-linux/pull/407/files looks rather complex to interface with.

files:

Something else?

A different way to approach this would be for Qubes to import the custom / derivative (however it can be called) nftables rules. So the firewall rules loading mechanism would be Qubes. Just the specific nftables rules additions would be template (Qubes-Whonix) specific.


Another way to phrase this: Unspecific to Qubes-Whonix...

How should derivatives load their custom additional nftables rules in Qubes?

Kinda similar to how should users load their custom additional nftables rules in Qubes?

(With a small detail that some users might want Qubes nftables rules + derivative additional nftables rules + custom additional nftables rules.)

1cho1ce commented 11 months ago

How should derivatives load their custom additional nftables rules in Qubes?

Kinda similar to how should users load their custom additional nftables rules in Qubes?

(With a small detail that some users might want Qubes nftables rules + derivative additional nftables rules + custom additional nftables rules.)

I think the proper way is to add custom rules in custom chains with priority higher or lower than the ones used by Qubes. For example, default ip qubes input type filter chain has priority filter (= 0) and if you want to add rules before it then create a new chain with priority decreased by 1: nft add chain ip qubes whonix-input '{ type filter hook input priority filter - 1; policy accept; }' And if you want to add rules after filter then create chain with priority increased by 1. nft add chain ip qubes whonix-input '{ type filter hook input priority filter + 1; policy accept; }' More info on priority: https://wiki.nftables.org/wiki-nftables/index.php/Netfilter_hooks#Priority_within_hook

But on the other hand if there will be a need for a multiple custom chains, for example, whonix-input and vpn-input then there is a question on how to handle them. If they will use the same priority then:

The priority is important since it determines the ordering of the chains, thus, if you have several chains in the input hook, you can decide which one sees packets before another. For example, input chains with priorities -12, -1, 0, 10 would be consulted exactly in that order. It's possible to give two base chains the same priority, but there is no guaranteed evaluation order of base chains with identical priority that are attached to the same hook location.

https://wiki.nftables.org/wiki-nftables/index.php/Configuring_chains

Maybe during the new custom chain creation it should search for unused priority nearest to the filter priority and use it. But it'll require to parse the nft list output which is not a good solution.

1cho1ce commented 11 months ago

But on the other hand if there will be a need for a multiple custom chains, for example, whonix-input and vpn-input then there is a question on how to handle them.

Maybe establish the firewall handling rules that custom chains must have specific naming. Like they should be named custom-pre-X-input and custom-post-X-input, where X - priority offset from Qubes input chain priority. And all the new custom chains should check for existing chains using this naming and find unused one with:

if nft create chain ip qubes custom-pre-$X-input '{ type filter hook input priority filter - $X; policy accept; }') &>/dev/null; then
...
fi
1cho1ce commented 11 months ago

The way it's done right now seems to be easier where you just insert or add jump to your custom chain in main chain: https://github.com/QubesOS/qubes-core-agent-linux/blob/main/network/qubes-ipv4.nft#L30 But there will be a problem with how to add chains that should be evaluated first. Using priorities, if Whonix chain should be evaluated before all (pre) or after all (post) other chains, then it can set priority with a gap: nft add chain ip qubes custom-pre-$((10+$X))-input '{ type filter hook input priority filter - $((10+$X)); policy accept; }' But for jumps to chains it'd require some parsing of nft list.

adrelanos commented 10 months ago

This ticket is a blocker for Whonix to be ported to nftables. This could use some guidance on how to implement this from Qubes developers.

@1cho1ce what you said is rather theoretic without looking at the actual Qubes and Whonix source code?

DemiMarie commented 10 months ago

This ticket is a blocker for Whonix to be ported to nftables. This could use some guidance on how to implement this from Qubes developers.

DemiMarie commented 10 months ago

Is this enough information @adrelanos?

1cho1ce commented 10 months ago

If you need additional chains in the qubes table (as jump or goto targets), prefix them (with e.g. whonix-) to avoid clashing with the rules provided by Qubes OS.

An example code snippet to create additional chain and insert jump rule to it in default Qubes OS custom-* chain only once on table creation:

if nft create chain ip qubes whonix-input '{ type filter hook input priority filter; policy accept; }') &>/dev/null; then
    nft add rule ip qubes custom-input jump whonix-input
fi
DemiMarie commented 10 months ago

@1cho1ce Better would be something like

nft 'add chain ip qubes whonix-input
flush chain ip qubes whonix-input
add rule ip qubes custom-input jump whonix-input'
adrelanos commented 10 months ago

Thank you, that's been helpful!

It will require more extensive changes than I was hoping but now I have something to work with.

I'll post here again should I have more questions.

1cho1ce commented 10 months ago

@1cho1ce Better would be something like

nft 'add chain ip qubes whonix-input
flush chain ip qubes whonix-input
add rule ip qubes custom-input jump whonix-input'

If it'll be called only once then it's a better choice indeed. But if you need to add/remove the custom chain, e.g. in case of VPN on VPN connect/disconnect, then you need to either make sure that you add jump rule in custom-input only once so it won't be duplicated multiple times or parse the custom-input to get the jump rule handler to remove this rule which is not good way.

DemiMarie commented 10 months ago

@1cho1ce Better would be something like

nft 'add chain ip qubes whonix-input
flush chain ip qubes whonix-input
add rule ip qubes custom-input jump whonix-input'

If it'll be called only once then it's a better choice indeed. But if you need to add/remove the custom chain, e.g. in case of VPN on VPN connect/disconnect, then you need to either make sure that you add jump rule in custom-input only once so it won't be duplicated multiple times or parse the custom-input to get the jump rule handler to remove this rule which is not good way.

This command is explicitly written to be idempotent and atomic. To delete the chain, just run:

nft 'delete rule ip qubes custom-input jump whonix-input
delete chain ip qubes whonix-input'
1cho1ce commented 10 months ago
nft 'delete rule ip qubes custom-input jump whonix-input
delete chain ip qubes whonix-input'
user@testqube:~$ sudo nft 'delete rule ip qubes custom-input jump whonix-input
delete chain ip qubes whonix-input'
Error: syntax error, unexpected jump, expecting handle
delete rule ip qubes custom-input jump whonix-input
                                  ^^^^

That's the problem, you can't delete rule using statement as it was with iptables. You can only delete it using its handle:

RULES
           {add | insert} rule [family] table chain [handle handle | index index] statement ... [comment comment]
           {delete | reset} rule [family] table chain handle handle

...

       delete rule from inet table.

           # nft -a list ruleset
           table inet filter {
                   chain input {
                           type filter hook input priority filter; policy accept;
                           ct state established,related accept # handle 4
                           ip saddr 10.1.1.1 tcp dport ssh accept # handle 5
                     ...
           # delete the rule with handle 5
           nft delete rule inet filter input handle 5
adrelanos commented 4 months ago

The deletion of Qubes' firewall rules has been fixed thanks to https://github.com/Whonix/whonix-firewall/pull/7

adrelanos commented 4 months ago

Since https://github.com/QubesOS/qubes-issues/issues/8398 was implemented, firewall rules could be simplified.


Here is an updated /var/lib/whonix-firewall/firewall.nft:

#!/usr/sbin/nft -f
add table inet nat
add table inet filter
add table ip6 nat
flush table inet nat
flush table inet filter
flush table ip6 nat
add chain inet filter input { type filter hook input priority 0; policy drop; }
add chain inet filter forward { type filter hook forward priority 0; policy drop; }
add chain inet filter output { type filter hook output priority 0; policy drop; }
add chain inet nat prerouting { type nat hook prerouting priority -100; }
add chain inet nat output { type nat hook output priority -100; }
add chain ip6 nat output { type nat hook output priority -100; }
add rule inet filter input ct state invalid counter drop
add rule inet filter input tcp flags & (fin|syn|rst|psh|ack|urg) == fin|syn|rst|ack counter drop
add rule inet filter input tcp flags & (fin|syn) == fin|syn counter drop
add rule inet filter input tcp flags & (syn|rst) == syn|rst counter drop
add rule inet filter input ip frag-off & 0x1fff != 0 counter drop
add rule inet filter input tcp flags & (fin|syn|rst|psh|ack|urg) == fin|syn|rst|psh|ack|urg counter drop
add rule inet filter input tcp flags & (fin|syn|rst|psh|ack|urg) == 0x0 counter drop
add rule inet filter input iifname lo counter accept
add rule inet filter input ct state established counter accept
add rule inet filter input icmp type destination-unreachable icmp code frag-needed ct state related counter accept
add rule inet filter input ip protocol icmp counter drop
add rule inet filter input ip6 nexthdr icmpv6 counter drop
add rule inet filter input iifname vif* udp dport 5300 counter accept
add rule inet filter input iifname vif* tcp dport 9040 counter accept
add rule inet filter input iifname vif* tcp dport 9051 counter accept
add rule inet filter input iifname vif* tcp dport 9050 counter accept
add rule inet filter input iifname vif* tcp dport 9100 counter accept
add rule inet filter input iifname vif* tcp dport 9101 counter accept
add rule inet filter input iifname vif* tcp dport 9102 counter accept
add rule inet filter input iifname vif* tcp dport 9103 counter accept
add rule inet filter input iifname vif* tcp dport 9104 counter accept
add rule inet filter input iifname vif* tcp dport 9105 counter accept
add rule inet filter input iifname vif* tcp dport 9106 counter accept
add rule inet filter input iifname vif* tcp dport 9107 counter accept
add rule inet filter input iifname vif* tcp dport 9108 counter accept
add rule inet filter input iifname vif* tcp dport 9109 counter accept
add rule inet filter input iifname vif* tcp dport 9110 counter accept
add rule inet filter input iifname vif* tcp dport 9111 counter accept
add rule inet filter input iifname vif* tcp dport 9114 counter accept
add rule inet filter input iifname vif* tcp dport 9115 counter accept
add rule inet filter input iifname vif* tcp dport 9117 counter accept
add rule inet filter input iifname vif* tcp dport 9118 counter accept
add rule inet filter input iifname vif* tcp dport 9122 counter accept
add rule inet filter input iifname vif* tcp dport 9123 counter accept
add rule inet filter input iifname vif* tcp dport 9124 counter accept
add rule inet filter input iifname vif* tcp dport 9125 counter accept
add rule inet filter input iifname vif* tcp dport 9150 counter accept
add rule inet filter input iifname vif* tcp dport 9152-9229 counter accept
add rule inet nat prerouting iifname vif* ip daddr 10.137.0.0/16 tcp dport 9051 counter redirect to :9051
add rule inet nat prerouting iifname vif* ip daddr 10.138.0.0/16 tcp dport 9051 counter redirect to :9051
add rule inet nat prerouting iifname vif* ip daddr 10.152.152.10 tcp dport 9051 counter redirect to :9051
add rule inet nat prerouting iifname vif* ip daddr 10.137.0.0/16 tcp dport 9050 counter redirect to :9050
add rule inet nat prerouting iifname vif* ip daddr 10.138.0.0/16 tcp dport 9050 counter redirect to :9050
add rule inet nat prerouting iifname vif* ip daddr 10.152.152.10 tcp dport 9050 counter redirect to :9050
add rule inet nat prerouting iifname vif* ip daddr 10.137.0.0/16 tcp dport 9100 counter redirect to :9100
add rule inet nat prerouting iifname vif* ip daddr 10.138.0.0/16 tcp dport 9100 counter redirect to :9100
add rule inet nat prerouting iifname vif* ip daddr 10.152.152.10 tcp dport 9100 counter redirect to :9100
add rule inet nat prerouting iifname vif* ip daddr 10.137.0.0/16 tcp dport 9101 counter redirect to :9101
add rule inet nat prerouting iifname vif* ip daddr 10.138.0.0/16 tcp dport 9101 counter redirect to :9101
add rule inet nat prerouting iifname vif* ip daddr 10.152.152.10 tcp dport 9101 counter redirect to :9101
add rule inet nat prerouting iifname vif* ip daddr 10.137.0.0/16 tcp dport 9102 counter redirect to :9102
add rule inet nat prerouting iifname vif* ip daddr 10.138.0.0/16 tcp dport 9102 counter redirect to :9102
add rule inet nat prerouting iifname vif* ip daddr 10.152.152.10 tcp dport 9102 counter redirect to :9102
add rule inet nat prerouting iifname vif* ip daddr 10.137.0.0/16 tcp dport 9103 counter redirect to :9103
add rule inet nat prerouting iifname vif* ip daddr 10.138.0.0/16 tcp dport 9103 counter redirect to :9103
add rule inet nat prerouting iifname vif* ip daddr 10.152.152.10 tcp dport 9103 counter redirect to :9103
add rule inet nat prerouting iifname vif* ip daddr 10.137.0.0/16 tcp dport 9104 counter redirect to :9104
add rule inet nat prerouting iifname vif* ip daddr 10.138.0.0/16 tcp dport 9104 counter redirect to :9104
add rule inet nat prerouting iifname vif* ip daddr 10.152.152.10 tcp dport 9104 counter redirect to :9104
add rule inet nat prerouting iifname vif* ip daddr 10.137.0.0/16 tcp dport 9105 counter redirect to :9105
add rule inet nat prerouting iifname vif* ip daddr 10.138.0.0/16 tcp dport 9105 counter redirect to :9105
add rule inet nat prerouting iifname vif* ip daddr 10.152.152.10 tcp dport 9105 counter redirect to :9105
add rule inet nat prerouting iifname vif* ip daddr 10.137.0.0/16 tcp dport 9106 counter redirect to :9106
add rule inet nat prerouting iifname vif* ip daddr 10.138.0.0/16 tcp dport 9106 counter redirect to :9106
add rule inet nat prerouting iifname vif* ip daddr 10.152.152.10 tcp dport 9106 counter redirect to :9106
add rule inet nat prerouting iifname vif* ip daddr 10.137.0.0/16 tcp dport 9107 counter redirect to :9107
add rule inet nat prerouting iifname vif* ip daddr 10.138.0.0/16 tcp dport 9107 counter redirect to :9107
add rule inet nat prerouting iifname vif* ip daddr 10.152.152.10 tcp dport 9107 counter redirect to :9107
add rule inet nat prerouting iifname vif* ip daddr 10.137.0.0/16 tcp dport 9108 counter redirect to :9108
add rule inet nat prerouting iifname vif* ip daddr 10.138.0.0/16 tcp dport 9108 counter redirect to :9108
add rule inet nat prerouting iifname vif* ip daddr 10.152.152.10 tcp dport 9108 counter redirect to :9108
add rule inet nat prerouting iifname vif* ip daddr 10.137.0.0/16 tcp dport 9109 counter redirect to :9109
add rule inet nat prerouting iifname vif* ip daddr 10.138.0.0/16 tcp dport 9109 counter redirect to :9109
add rule inet nat prerouting iifname vif* ip daddr 10.152.152.10 tcp dport 9109 counter redirect to :9109
add rule inet nat prerouting iifname vif* ip daddr 10.137.0.0/16 tcp dport 9110 counter redirect to :9110
add rule inet nat prerouting iifname vif* ip daddr 10.138.0.0/16 tcp dport 9110 counter redirect to :9110
add rule inet nat prerouting iifname vif* ip daddr 10.152.152.10 tcp dport 9110 counter redirect to :9110
add rule inet nat prerouting iifname vif* ip daddr 10.137.0.0/16 tcp dport 9111 counter redirect to :9111
add rule inet nat prerouting iifname vif* ip daddr 10.138.0.0/16 tcp dport 9111 counter redirect to :9111
add rule inet nat prerouting iifname vif* ip daddr 10.152.152.10 tcp dport 9111 counter redirect to :9111
add rule inet nat prerouting iifname vif* ip daddr 10.137.0.0/16 tcp dport 9114 counter redirect to :9114
add rule inet nat prerouting iifname vif* ip daddr 10.138.0.0/16 tcp dport 9114 counter redirect to :9114
add rule inet nat prerouting iifname vif* ip daddr 10.152.152.10 tcp dport 9114 counter redirect to :9114
add rule inet nat prerouting iifname vif* ip daddr 10.137.0.0/16 tcp dport 9115 counter redirect to :9115
add rule inet nat prerouting iifname vif* ip daddr 10.138.0.0/16 tcp dport 9115 counter redirect to :9115
add rule inet nat prerouting iifname vif* ip daddr 10.152.152.10 tcp dport 9115 counter redirect to :9115
add rule inet nat prerouting iifname vif* ip daddr 10.137.0.0/16 tcp dport 9117 counter redirect to :9117
add rule inet nat prerouting iifname vif* ip daddr 10.138.0.0/16 tcp dport 9117 counter redirect to :9117
add rule inet nat prerouting iifname vif* ip daddr 10.152.152.10 tcp dport 9117 counter redirect to :9117
add rule inet nat prerouting iifname vif* ip daddr 10.137.0.0/16 tcp dport 9118 counter redirect to :9118
add rule inet nat prerouting iifname vif* ip daddr 10.138.0.0/16 tcp dport 9118 counter redirect to :9118
add rule inet nat prerouting iifname vif* ip daddr 10.152.152.10 tcp dport 9118 counter redirect to :9118
add rule inet nat prerouting iifname vif* ip daddr 10.137.0.0/16 tcp dport 9122 counter redirect to :9122
add rule inet nat prerouting iifname vif* ip daddr 10.138.0.0/16 tcp dport 9122 counter redirect to :9122
add rule inet nat prerouting iifname vif* ip daddr 10.152.152.10 tcp dport 9122 counter redirect to :9122
add rule inet nat prerouting iifname vif* ip daddr 10.137.0.0/16 tcp dport 9123 counter redirect to :9123
add rule inet nat prerouting iifname vif* ip daddr 10.138.0.0/16 tcp dport 9123 counter redirect to :9123
add rule inet nat prerouting iifname vif* ip daddr 10.152.152.10 tcp dport 9123 counter redirect to :9123
add rule inet nat prerouting iifname vif* ip daddr 10.137.0.0/16 tcp dport 9124 counter redirect to :9124
add rule inet nat prerouting iifname vif* ip daddr 10.138.0.0/16 tcp dport 9124 counter redirect to :9124
add rule inet nat prerouting iifname vif* ip daddr 10.152.152.10 tcp dport 9124 counter redirect to :9124
add rule inet nat prerouting iifname vif* ip daddr 10.137.0.0/16 tcp dport 9125 counter redirect to :9125
add rule inet nat prerouting iifname vif* ip daddr 10.138.0.0/16 tcp dport 9125 counter redirect to :9125
add rule inet nat prerouting iifname vif* ip daddr 10.152.152.10 tcp dport 9125 counter redirect to :9125
add rule inet nat prerouting iifname vif* ip daddr 10.137.0.0/16 tcp dport 9150 counter redirect to :9150
add rule inet nat prerouting iifname vif* ip daddr 10.138.0.0/16 tcp dport 9150 counter redirect to :9150
add rule inet nat prerouting iifname vif* ip daddr 10.152.152.10 tcp dport 9150 counter redirect to :9150
add rule inet nat prerouting iifname vif* ip daddr 10.137.0.0/16 tcp dport 9152-9229 counter redirect
add rule inet nat prerouting iifname vif* ip daddr 10.138.0.0/16 tcp dport 9152-9229 counter redirect
add rule inet nat prerouting iifname vif* ip daddr 10.152.152.10 tcp dport 9152-9229 counter redirect
add rule inet nat prerouting iifname vif* udp dport 53 counter redirect to :5300
add rule inet nat prerouting iifname vif* tcp flags & (fin|syn|rst|ack) == syn counter redirect to :9040
add rule inet filter input counter drop
add rule inet filter forward counter reject
add rule inet filter output ct state invalid counter reject
add rule inet filter output ct state invalid counter reject
add rule inet filter output tcp flags & (fin|syn|rst|psh|ack|urg) == fin|syn|rst|ack counter reject
add rule inet filter output tcp flags & (fin|syn) == fin|syn counter reject
add rule inet filter output tcp flags & (syn|rst) == syn|rst counter reject
add rule inet filter output ip frag-off & 0x1fff != 0 counter reject
add rule inet filter output tcp flags & (fin|syn|rst|psh|ack|urg) == fin|syn|rst|psh|ack|urg counter reject
add rule inet filter output tcp flags & (fin|syn|rst|psh|ack|urg) == 0x0 counter reject
add rule inet nat output skuid 104 counter return
add rule inet nat output skuid 102 counter return
add rule inet nat output skuid 101 counter return
add rule inet filter output ct state established counter accept
add rule inet filter output oifname lo counter accept
add rule inet filter output skuid 104 counter accept
add rule inet filter output skuid 102 counter accept
add rule inet filter output skuid 101 counter accept
add rule inet filter output counter reject

Here is the sudo nft --stateless list ruleset:

table ip qubes {
    set downstream {
        type ipv4_addr
        elements = { 10.137.0.86 }
    }

    set allowed {
        type ifname . ipv4_addr
        elements = { "vif20.0" . 10.137.0.86 }
    }

    chain prerouting {
        type filter hook prerouting priority raw; policy accept;
        iifgroup 2 goto antispoof
        ip saddr @downstream counter drop
    }

    chain antispoof {
        iifname . ip saddr @allowed accept
        counter drop
    }
}
table ip6 qubes {
    set downstream {
        type ipv6_addr
    }

    set allowed {
        type ifname . ipv6_addr
    }

    chain antispoof {
        iifname . ip6 saddr @allowed accept
        counter drop
    }

    chain prerouting {
        type filter hook prerouting priority raw; policy accept;
        iifgroup 2 goto antispoof
        ip6 saddr @downstream counter drop
    }
}
table inet nat {
    chain prerouting {
        type nat hook prerouting priority dstnat; policy accept;
        iifname "vif*" ip daddr 10.137.0.0/16 tcp dport 9051 counter redirect to :9051
        iifname "vif*" ip daddr 10.138.0.0/16 tcp dport 9051 counter redirect to :9051
        iifname "vif*" ip daddr 10.152.152.10 tcp dport 9051 counter redirect to :9051
        iifname "vif*" ip daddr 10.137.0.0/16 tcp dport 9050 counter redirect to :9050
        iifname "vif*" ip daddr 10.138.0.0/16 tcp dport 9050 counter redirect to :9050
        iifname "vif*" ip daddr 10.152.152.10 tcp dport 9050 counter redirect to :9050
        iifname "vif*" ip daddr 10.137.0.0/16 tcp dport 9100 counter redirect to :9100
        iifname "vif*" ip daddr 10.138.0.0/16 tcp dport 9100 counter redirect to :9100
        iifname "vif*" ip daddr 10.152.152.10 tcp dport 9100 counter redirect to :9100
        iifname "vif*" ip daddr 10.137.0.0/16 tcp dport 9101 counter redirect to :9101
        iifname "vif*" ip daddr 10.138.0.0/16 tcp dport 9101 counter redirect to :9101
        iifname "vif*" ip daddr 10.152.152.10 tcp dport 9101 counter redirect to :9101
        iifname "vif*" ip daddr 10.137.0.0/16 tcp dport 9102 counter redirect to :9102
        iifname "vif*" ip daddr 10.138.0.0/16 tcp dport 9102 counter redirect to :9102
        iifname "vif*" ip daddr 10.152.152.10 tcp dport 9102 counter redirect to :9102
        iifname "vif*" ip daddr 10.137.0.0/16 tcp dport 9103 counter redirect to :9103
        iifname "vif*" ip daddr 10.138.0.0/16 tcp dport 9103 counter redirect to :9103
        iifname "vif*" ip daddr 10.152.152.10 tcp dport 9103 counter redirect to :9103
        iifname "vif*" ip daddr 10.137.0.0/16 tcp dport 9104 counter redirect to :9104
        iifname "vif*" ip daddr 10.138.0.0/16 tcp dport 9104 counter redirect to :9104
        iifname "vif*" ip daddr 10.152.152.10 tcp dport 9104 counter redirect to :9104
        iifname "vif*" ip daddr 10.137.0.0/16 tcp dport 9105 counter redirect to :9105
        iifname "vif*" ip daddr 10.138.0.0/16 tcp dport 9105 counter redirect to :9105
        iifname "vif*" ip daddr 10.152.152.10 tcp dport 9105 counter redirect to :9105
        iifname "vif*" ip daddr 10.137.0.0/16 tcp dport 9106 counter redirect to :9106
        iifname "vif*" ip daddr 10.138.0.0/16 tcp dport 9106 counter redirect to :9106
        iifname "vif*" ip daddr 10.152.152.10 tcp dport 9106 counter redirect to :9106
        iifname "vif*" ip daddr 10.137.0.0/16 tcp dport 9107 counter redirect to :9107
        iifname "vif*" ip daddr 10.138.0.0/16 tcp dport 9107 counter redirect to :9107
        iifname "vif*" ip daddr 10.152.152.10 tcp dport 9107 counter redirect to :9107
        iifname "vif*" ip daddr 10.137.0.0/16 tcp dport 9108 counter redirect to :9108
        iifname "vif*" ip daddr 10.138.0.0/16 tcp dport 9108 counter redirect to :9108
        iifname "vif*" ip daddr 10.152.152.10 tcp dport 9108 counter redirect to :9108
        iifname "vif*" ip daddr 10.137.0.0/16 tcp dport 9109 counter redirect to :9109
        iifname "vif*" ip daddr 10.138.0.0/16 tcp dport 9109 counter redirect to :9109
        iifname "vif*" ip daddr 10.152.152.10 tcp dport 9109 counter redirect to :9109
        iifname "vif*" ip daddr 10.137.0.0/16 tcp dport 9110 counter redirect to :9110
        iifname "vif*" ip daddr 10.138.0.0/16 tcp dport 9110 counter redirect to :9110
        iifname "vif*" ip daddr 10.152.152.10 tcp dport 9110 counter redirect to :9110
        iifname "vif*" ip daddr 10.137.0.0/16 tcp dport 9111 counter redirect to :9111
        iifname "vif*" ip daddr 10.138.0.0/16 tcp dport 9111 counter redirect to :9111
        iifname "vif*" ip daddr 10.152.152.10 tcp dport 9111 counter redirect to :9111
        iifname "vif*" ip daddr 10.137.0.0/16 tcp dport 9114 counter redirect to :9114
        iifname "vif*" ip daddr 10.138.0.0/16 tcp dport 9114 counter redirect to :9114
        iifname "vif*" ip daddr 10.152.152.10 tcp dport 9114 counter redirect to :9114
        iifname "vif*" ip daddr 10.137.0.0/16 tcp dport 9115 counter redirect to :9115
        iifname "vif*" ip daddr 10.138.0.0/16 tcp dport 9115 counter redirect to :9115
        iifname "vif*" ip daddr 10.152.152.10 tcp dport 9115 counter redirect to :9115
        iifname "vif*" ip daddr 10.137.0.0/16 tcp dport 9117 counter redirect to :9117
        iifname "vif*" ip daddr 10.138.0.0/16 tcp dport 9117 counter redirect to :9117
        iifname "vif*" ip daddr 10.152.152.10 tcp dport 9117 counter redirect to :9117
        iifname "vif*" ip daddr 10.137.0.0/16 tcp dport 9118 counter redirect to :9118
        iifname "vif*" ip daddr 10.138.0.0/16 tcp dport 9118 counter redirect to :9118
        iifname "vif*" ip daddr 10.152.152.10 tcp dport 9118 counter redirect to :9118
        iifname "vif*" ip daddr 10.137.0.0/16 tcp dport 9122 counter redirect to :9122
        iifname "vif*" ip daddr 10.138.0.0/16 tcp dport 9122 counter redirect to :9122
        iifname "vif*" ip daddr 10.152.152.10 tcp dport 9122 counter redirect to :9122
        iifname "vif*" ip daddr 10.137.0.0/16 tcp dport 9123 counter redirect to :9123
        iifname "vif*" ip daddr 10.138.0.0/16 tcp dport 9123 counter redirect to :9123
        iifname "vif*" ip daddr 10.152.152.10 tcp dport 9123 counter redirect to :9123
        iifname "vif*" ip daddr 10.137.0.0/16 tcp dport 9124 counter redirect to :9124
        iifname "vif*" ip daddr 10.138.0.0/16 tcp dport 9124 counter redirect to :9124
        iifname "vif*" ip daddr 10.152.152.10 tcp dport 9124 counter redirect to :9124
        iifname "vif*" ip daddr 10.137.0.0/16 tcp dport 9125 counter redirect to :9125
        iifname "vif*" ip daddr 10.138.0.0/16 tcp dport 9125 counter redirect to :9125
        iifname "vif*" ip daddr 10.152.152.10 tcp dport 9125 counter redirect to :9125
        iifname "vif*" ip daddr 10.137.0.0/16 tcp dport 9150 counter redirect to :9150
        iifname "vif*" ip daddr 10.138.0.0/16 tcp dport 9150 counter redirect to :9150
        iifname "vif*" ip daddr 10.152.152.10 tcp dport 9150 counter redirect to :9150
        iifname "vif*" ip daddr 10.137.0.0/16 tcp dport 9152-9229 counter redirect
        iifname "vif*" ip daddr 10.138.0.0/16 tcp dport 9152-9229 counter redirect
        iifname "vif*" ip daddr 10.152.152.10 tcp dport 9152-9229 counter redirect
        iifname "vif*" udp dport 53 counter redirect to :5300
        iifname "vif*" tcp flags syn / fin,syn,rst,ack counter redirect to :9040
    }

    chain output {
        type nat hook output priority -100; policy accept;
        meta skuid 104 counter return
        meta skuid 102 counter return
        meta skuid 101 counter return
    }
}
table inet filter {
    chain input {
        type filter hook input priority filter; policy drop;
        ct state invalid counter drop
        tcp flags fin,syn,rst,ack / fin,syn,rst,psh,ack,urg counter drop
        tcp flags fin,syn / fin,syn counter drop
        tcp flags syn,rst / syn,rst counter drop
        ip frag-off & 8191 != 0 counter drop
        tcp flags fin,syn,rst,psh,ack,urg / fin,syn,rst,psh,ack,urg counter drop
        tcp flags ! fin,syn,rst,psh,ack,urg counter drop
        iifname "lo" counter accept
        ct state established counter accept
        icmp type destination-unreachable icmp code frag-needed ct state related counter accept
        ip protocol icmp counter drop
        ip6 nexthdr ipv6-icmp counter drop
        iifname "vif*" udp dport 5300 counter accept
        iifname "vif*" tcp dport 9040 counter accept
        iifname "vif*" tcp dport 9051 counter accept
        iifname "vif*" tcp dport 9050 counter accept
        iifname "vif*" tcp dport 9100 counter accept
        iifname "vif*" tcp dport 9101 counter accept
        iifname "vif*" tcp dport 9102 counter accept
        iifname "vif*" tcp dport 9103 counter accept
        iifname "vif*" tcp dport 9104 counter accept
        iifname "vif*" tcp dport 9105 counter accept
        iifname "vif*" tcp dport 9106 counter accept
        iifname "vif*" tcp dport 9107 counter accept
        iifname "vif*" tcp dport 9108 counter accept
        iifname "vif*" tcp dport 9109 counter accept
        iifname "vif*" tcp dport 9110 counter accept
        iifname "vif*" tcp dport 9111 counter accept
        iifname "vif*" tcp dport 9114 counter accept
        iifname "vif*" tcp dport 9115 counter accept
        iifname "vif*" tcp dport 9117 counter accept
        iifname "vif*" tcp dport 9118 counter accept
        iifname "vif*" tcp dport 9122 counter accept
        iifname "vif*" tcp dport 9123 counter accept
        iifname "vif*" tcp dport 9124 counter accept
        iifname "vif*" tcp dport 9125 counter accept
        iifname "vif*" tcp dport 9150 counter accept
        iifname "vif*" tcp dport 9152-9229 counter accept
        counter drop
    }

    chain forward {
        type filter hook forward priority filter; policy drop;
        counter reject
    }

    chain output {
        type filter hook output priority filter; policy drop;
        ct state invalid counter reject
        ct state invalid counter reject
        tcp flags fin,syn,rst,ack / fin,syn,rst,psh,ack,urg counter reject
        tcp flags fin,syn / fin,syn counter reject
        tcp flags syn,rst / syn,rst counter reject
        ip frag-off & 8191 != 0 counter reject with icmp port-unreachable
        tcp flags fin,syn,rst,psh,ack,urg / fin,syn,rst,psh,ack,urg counter reject
        tcp flags ! fin,syn,rst,psh,ack,urg counter reject
        ct state established counter accept
        oifname "lo" counter accept
        meta skuid 104 counter accept
        meta skuid 102 counter accept
        meta skuid 101 counter accept
        counter reject
    }
}
table ip6 nat {
    chain output {
        type nat hook output priority -100; policy accept;
    }
}
table inet qubes-nat-accel {
    flowtable qubes-accel {
        hook ingress priority filter
        devices = { eth0, eth1, lo, vif20.0 }
    }

    chain qubes-accel {
        type filter hook forward priority filter + 5; policy accept;
        meta l4proto { tcp, udp } iifgroup 2 oifgroup 1 flow add @qubes-accel
        counter
    }
}

Qubes' default nftables rules are no longer removed. In other words, none of the nftables rules set by Qubes are missing.