Open alan-wint opened 1 year ago
ktsakalozos
commented
1 year ago Hi @alan-wint. Thank you for bringing this up. I am sure you have seen that this is a known k8s issue [1]. Please allow us some time to see if/how we can address it in an elegant way.
[1] https://kubernetes.io/docs/tasks/administer-cluster/dns-debugging-resolution/#known-issues
phpwutz
commented
1 year ago thie becomes even more lovely now that systemd adds the ipv6 versions too, so having 2 DNS servers results in 4 entries triggering this warning :/
my current workaround is to have a /etc/resolv-static.conf and add --resolv-conf=/etc/resolv-static.conf to /var/snap/microk8s/current/args/kubelet but this is really not very satisfying
rahul799
commented
1 year ago Hi @ktsakalozos, do we have any workarounds here? It's kind of blocking our deployments. It will be great if you can help.
RobT2012
commented
1 year ago Late reply, I also have a similar issue and was hoping that the systemd version resolved the 3 limit issue. What I've had to do, is use dnsmasq to get around the problem. In many ways dnsmasq provides a better service and even if the 3 limit is resolved, I'll probably continue to use dnsmasq.
To fix the 3 limit with dnsmasq,
1) install dnsmasq
Now configure dnsmaq
2) Edit /etc/resolv.conf, it should ONLY have these 2 lines Note: if systemd-resolv is used then edit the conf file it uses if it is not symlinked as /etc/resolv.conf
nameserver ::1 nameserver 127.0.0.1
3) touch /etc/resolv-dnsmasq.conf Note: this file can go anywhere and can be named whatever you want
Add the following lines replace with IP addrs to your local DNS's:
nameserver 2001:xxx:xxx:xxx:xxx:xxx nameserver 192.168.xxx.xxx (repeat for all your local DNS's - the limit goes past only 3)
4) touch /etc/dnsmasq.d/custom-settings.conf Note: this file can be named whatever you want
Add these lines, and adjust according to your preferences
point to the file created in step 3 and use local interface
resolv-file=/etc/resolv-dnsmasq.conf interface=lo bind-interfaces all-servers cache-size=0 no-negcache
5) systemctl restart dnsmasq
Done!
Note: I also use dnsmasq for my local DNS servers that speak to external authoritative servers such as 8.8.8.8, 8.8.4.4 etc. The configuration is similar, but of course more complicated if you have local services such as DHCP and internal names resolution, etc, but dnsmasq will do the job very well and I've not encountered any frustrating limits.
adrian-moisa
commented
8 months ago Rolling back to an older version helped me. Full explanation here: Github: kube-proxy pods continuously CrashLoopBackOff #118461
sudo apt-get install -y kubelet=1.23.17-00 kubeadm=1.23.17-00 kubectl=1.23.17-00 on Ubuntu LTS 22.04
bitemarcz
commented
7 months ago I tried the dnsmasq solution but no luck. Seems to have made some parts of this work or at least got me past the limits issue but the Calico and Core DNS pods are still showing pod sandbox changed, it will be killed and re-created.
I tried killing the pods and restarting but appears to get me no where. Trying to see if there's any suggestions here.
splitice
commented
3 months ago coredns detects dnsmasq as a loop therefore aborts. It might be possible to run dnsmasq on 127.0.0.2 to avoid this but I havent tested it
Summary
I have more than 3 DNS servers configured in resolv.conf and that is a hard requirement, also not unusual afaik. My syslog is full of "Nameserver limits exceeded" messages.
What Should Happen Instead?
There should be no problem with more than three DNS servers.
Reproduction Steps
/etc/resolv.confor via/etc/systemd/resolved.confIntrospection Report
Skipped as it contains private info that is not for a public forum like this. Example log line:
microk8s.daemon-kubelite[3713]: E0221 17:00:05.944148 3713 dns.go:156] "Nameserver limits exceeded" err="Nameserver limits were exce eded, some nameservers have been omitted, the applied nameserver line is: 7.7.8.9 143.212.212.212 8660:fc::fc"Can you suggest a fix?
Are you interested in contributing with a fix?
No