Open lion7 opened 4 days ago
smira
commented
3 days ago DNS requests are forwarded to the host using a hardcoded 169.254.116.108 IPv4 address, but there is no matching IPv4 route in a IPv6-only environment.
This one I'm confused a bit, as the address is assigned to the host network. Or you mean that CoreDNS with pod networking doesn't have IPv4 address at all, so it can't reach out?
lion7
commented
3 days ago DNS requests are forwarded to the host using a hardcoded 169.254.116.108 IPv4 address, but there is no matching IPv4 route in a IPv6-only environment.
This one I'm confused a bit, as the address is assigned to the host network. Or you mean that CoreDNS with pod networking doesn't have IPv4 address at all, so it can't reach out?
Sorry, should have mentioned that indeed my PodCIDR is IPv6 only. So the pods only have an IPv6 address assigned. My cluster is basically IPv6 single stack instead of dual-stack.
lion7
commented
3 days ago Something else that might be of interest is that I'm using the bridge CNI plugin that comes bundled with Talos. Here is my CNI configuration:
❯ talosctl cat /etc/cni/net.d/bridge-cni.conflist
{
"cniVersion": "1.0.0",
"name": "cbr0",
"plugins": [
{
"type": "bridge",
"ipam": {
"type": "host-local",
"subnet": "2a02:****::****::****::c:0/120"
},
"dns": {},
"isDefaultGateway": true
}
]
}Since I'm using a GUA IPv6 prefix the pods can directly access the internet without any need for NAT / masquerading / etc. So from a networking perspective this is I guess the simplest setup that you can get.
Also, since I have no firewall configured (yet) I can actually reach the pods directly from any other server in the world.
Sidenote: to generate the above CNI configuration I had to use a Daemonset to generate and write it to /etc/cni/net.d which was mounted using a hostPath volume (https://github.com/lion7/bridge-cni).
It would be nice if we could write files to /etc/cni/net.d directly from the machine configuration (right now that path is prohibited from being used). Especially now that a few default CNI plugins are bundled with Talos.
Bug Report
Description
DNS not working when Talos is installed on a machine in an IPv6-only network. I needed to do multiple customizations and workarounds which I think should not be needed.
Some things that I noticed:
Default nameservers
Issue: The default nameservers are
1.1.1.1and8.8.8.8, which cannot be reached from IPv6.Workaround: Configure an IPv6 nameserver using the dashboard (requires a local screen/keyboard or BMC). Alternatively configure a DNS server using the
ip=:::::::<dns0-ip>:<dns1-ip>:<ntp0-ip>kernel arg.Better solution: Talos should use the IPv6 equivalent when no IPv4 address is configured on any interface.
Nameservers announced by ND are ignored
Issue: DNS servers that are announced via ND (SLAAC setup) are currently ignored by Talos.
Workaround: Override DNS servers using machine configuration.
Better solution: Talso should use the announced DNS servers from ND similar to how it does for DNS servers announced via IPv4 DHCP.
IPv6 endpoint for Image Factory
Issue: Talos Image Factory (
factory.talos.dev) has no IPv6 addresses configured.Workaround: Run a pull-through registry on a dual-stack server.
Better solution: Already tracked by https://github.com/siderolabs/image-factory/issues/60
DNS forwarding to host
Issue: DNS requests are forwarded to the host using a hardcoded
169.254.116.108IPv4 address, but there is no matching IPv4 route in a IPv6-only environment.Workaround: Disable DNS forwarding.
Better solution: Also use a hardcoded IPv6 address as a fallback.
Logs
N/A
Output of
talosctl get routes | grep -v vethEnvironment
metal