search

systemd / systemd

The systemd System and Service Manager
https://systemd.io
GNU General Public License v2.0
13.05k stars 3.74k forks source link

VRF Enablement #28840

Open VariableDeclared opened 1 year ago

VariableDeclared commented 1 year ago

Component

systemd-resolved

Is your feature request related to a problem? Please describe

When enabling VRFs DNS resolution will only work on the default VRF with systemd-resolved enabled

Describe the solution you'd like

I'm not sure what the correct solution is here, this bug is to document the limitation of systemd-resolved. Davind Ahern has a great writeup on it - https://people.kernel.org/dsahern/management-vrf-and-dns

Describe alternatives you've considered

Disable systemd-resolved, and just point /etc/resolv.conf to upstream Leave systemd-resolved enabled, only having DNS resolution on the default VRF

The systemd version you checked that didn't have the feature you are asking for

249.11

poettering commented 1 year ago

I cannot parse this. Any client can talk to resolved, it just answers universal truths about IP addresses/names.

Or do you want per-VRF DNS zones with different dns servers?

VariableDeclared commented 9 months ago

To be clear - i dont have a strong opinion or idea right now what the correct solution might be. Potentially systemd-resolved has no role to play. Simply raising this.

Jamesits commented 3 months ago

systemd-resolved on a VRF-enabled system works in a not very intuitive way.

For example, when all your network connections are in different VRFs (a very typical configuration on an ISP core router):

# empty main routing table
root@router:~# ip -4 route show

# Internet in VRF
root@router:~# ip -4 route show vrf isp1
default via 192.0.2.1 dev ens256 proto static
192.0.2.0/24 dev ens256 proto kernel scope link src 192.0.2.2

# the local table, which will become relevant later
root@router:~# ip -4 route show table local
local 127.0.0.0/8 dev lo proto kernel scope host src 127.0.0.1 
local 127.0.0.1 dev lo proto kernel scope host src 127.0.0.1 
broadcast 127.255.255.255 dev lo proto kernel scope link src 127.0.0.1

and resolved correctly reads the DNS config from networkd: (ens256 is the uplink port, isp1 is the corresponding VRF loopback/L3 master interface)

root@router:~# resolvectl
Global
       Protocols: +LLMNR +mDNS -DNSOverTLS DNSSEC=no/unsupported
resolv.conf mode: stub

Link 4 (ens256)
    Current Scopes: DNS LLMNR/IPv4 LLMNR/IPv6
         Protocols: +DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
Current DNS Server: 1.1.1.1
       DNS Servers: 1.1.1.1 1.0.0.1

Link 5 (isp1)
Current Scopes: none
     Protocols: -DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported

A user would expect local DNS resolution to be working inside the VRF but not in main; but in reality, DNS resolution works in main but not in VRF.

root@router:~# host google.com
google.com has address 172.217.24.238
google.com has IPv6 address 2404:6800:4005:814::200e
google.com mail is handled by 10 smtp.google.com.

root@router:~# ip vrf exec isp1 host google.com
;; communications error to 127.0.0.53#53: timed out
;; communications error to 127.0.0.53#53: timed out
;; no servers could be reached

This is caused by systemd-resolved listening explicitly on 127.0.0.53%lo:53, thus unable to receive any requests from inside a VRF. (It correctly sends the DNS request to the upstream servers since binding the socket on a interface will correctly attach the socket to the linked VRF.)

Linux do try to workaround this issue by using a PBR rule to redirect all local traffic to the local routing table:

root@router:~# ip rule
0:      from all lookup local
1000:   from all lookup [l3mdev-table]
32765:  from all fwmark 0x3e9 lookup 1001 proto static # isp1 table
32766:  from all lookup main
32767:  from all lookup default

But ip vrf exec uses a BPF program to redirect all the sockets of a program which rendered this workaround invalid. Also it doesn't work even if net.ipv4.tcp_l3mdev_accept = 1 and net.ipv4.udp_l3mdev_accept = 1 is set because of the explicit interface bind.

Here I'd like to propose 2 changes to make systemd-resolved work as a normal user would expect in this situation: