Qubes-firewall applies incorrect DNS rules when using VPN and limiting AppVM connections

[tezeb]

Qubes OS release

4.1

Brief summary

I searched the issues and I believe this one to be distinct than other possibly related (ie. https://github.com/QubesOS/qubes-issues/issues/1183, https://github.com/QubesOS/qubes-issues/issues/5706, https://github.com/QubesOS/qubes-issues/issues/6927).

With the following setup: AppVM --> VPN VM --> Firewall VM --> NetVM and with the "AppVM" firewall set to limit outgoing connections to set of hosts(via GUI), the firewall in VPN VM applies DNS rules with incorrect DNS servers set, thus cutting off the actual networking in AppVM.

For the AppVM with ip (10.137.0.27), that should be able to connect only to "X.X.X.X", the rules looks like: sudo nft list table qubes-firewall

        chain qbs-10-137-0-27 {
                ip daddr X.X.X.X tcp dport 443 accept
                ip daddr 127.0.0.53 tcp dport 53 accept
                ip daddr 127.0.0.53 udp dport 53 accept
                ip protocol icmp accept
                reject with icmp admin-prohibited
                reject with icmp admin-prohibited
        }

and the sudo iptables -t nat -nvL shows:

Chain PR-QBS (1 references)
 pkts bytes target     prot opt in     out     source               destination         
  770 54160 DNAT       udp  --  *      *       0.0.0.0/0            10.139.1.1           udp dpt:53 to:1.1.1.1
    0     0 DNAT       tcp  --  *      *       0.0.0.0/0            10.139.1.1           tcp dpt:53 to:1.1.1.1
   64  4304 DNAT       udp  --  *      *       0.0.0.0/0            10.139.1.2           udp dpt:53 to:8.8.8.8
    0     0 DNAT       tcp  --  *      *       0.0.0.0/0            10.139.1.2           tcp dpt:53 to:8.8.8.8

It is my understanding that the script /etc/dhclient.d/qubes-setup-dnat-to-ns.sh works correctly, but the qubes-firewall uses the incorrect values to setup nft rules, not respecting that the systemd-resolved is active using the value from /etc/resolv.conf instead.

This used to work fine in 4.0 as far as I can tell, as the VMs in question were migrated directly from a working installation of 4.0. The problem affects migrated as well as newly created AppVMs.

Steps to reproduce

1. Create AppVM and configure it's firewall to "Limit outgoing connections" with a one or more hosts.
2. Configure AppVM to use VPN-VM, which uses custom DNS(vpn-provided).
3. Try connecting from AppVM to the allowed host(using domain name).

Expected behavior

AppVM connects succesfully to the allowed host.

Actual behavior

It is not possible to resolve the domain name.

[tezeb]

If my understanding is correct, the dns_addresses function of firewall setup script should be enhanced and use the same mechanism that is used by dhclient support script which in this case relies on getting dns from resolved.

This seems as the correct way forward according to the commit message that created get-dns-from-resolved script:

When a qube provides network to others, it needs to forward DNS traffic.
If /etc/resolv.conf points at local systemd-resolved, redirecting to it
won't work (and is not a good idea). Instead, forward such queries to
systemd-resolved's upstream servers.

[brycepg]

I'm having a similar issue with DNS in 4.1.1 - My issue only applies starting with a vpn AppVM that has a template of fedora 34 or fedora 35 (tested to not work). For some reason when I set the fedora template to fedora-32 or fedora-33 for the vpn AppVM, dns resolves properly and openvpn works fine. Changing the template for sys-net and sys-firewall doesn't break the dns.

[3hhh]

With the following setup: AppVM --> VPN VM --> Firewall VM --> NetVM

That architecture is pretty bad practice as Qubes will always use the next downstream VM to enforce your firewall rules, i.e. in your case the VPN VM. This may mix up rules with your VPN client or whatever. So try to add another firewall VM in front of the VPN VM instead, i.e. AppVM --> Firewall VM 2 --> VPN VM --> Firewall VM --> NetVM.

[tezeb]

@3hhh please explain why is that a bad practice? What VPN client rules are you referring to? It used to work until very recently. The change that brought this is incompatible with dhclient support script. Please see my second comment for details.