nextdns / nextdns

NextDNS CLI client (DoH Proxy)
https://nextdns.io
MIT License
3.17k stars 191 forks source link

High CPU usage on USG when using Home Assistant #886

Closed taglia closed 7 months ago

taglia commented 10 months ago

Context

Description I have recently noticed that the CPU usage on my USG would rapidly increase, to the point where I start having network issues unless I restart NextDNS on the USG (the memory usage does not increase noticeably though). The problem appeared when I started running Home Assistant (I am running Core version 2023.11.2). I suspect this is linked to Home Assistant also because, if I run nextdns discovered, the output is flooded by homeassistant???????.local. records. Roughly one hour after a nextdns restart, I found 17,760 such records:

"homeassistant???????.local.": [
            "192.168.???.???"
        ],

This only happens on the subnet I use for IoT devices, the other IP addresses that Home Assistant uses do not appear, so it could be linked to some particularly chatty integration.

Config

admin@USGRouter:~$ nextdns config
debug false
control /var/run/nextdns.sock
cache-max-age 0s
mdns all
timeout 5s
setup-router true
log-queries false
cache-size 10MB
max-ttl 5s
report-client-info true
discovery-dns 
detect-captive-portals false
use-hosts true
listen localhost:53
profile 192.168.X.Y/24=??????
profile ??????
hardened-privacy false
bogus-priv true
max-inflight-requests 256
auto-activate true
admin@USGRouter:~$ 

I am using two different NextDNS profiles (which is the only config I customized), but I already tried to remove one, and it did not make any difference about this problem. I have not seen anything abnormal in NextDNS logs.

For now I will uninstall NextDNS from my USG, but I would be happy to do more investigation if that is useful.

justfalter commented 8 months ago

I ran into this issue, as well. I have not yet figured out why Home Assistant is constantly advertising new names.

I did find that disabling mdns discovery in nextdns helped out: nextdns config set -mdns disabled.

If there were anything at fault with the nextdns CLI, perhaps mdns records should expire after some amount of time? At the moment, it appears that nextdns holds on to all mdns hosts forever.

rs commented 8 months ago

I have a rather large home assistant installation with a UDM SE running the CLI at home and can’t reproduce your issue.

justfalter commented 8 months ago

The issue seems to present itself when:

  1. Home Assistant has two (or more) network interfaces, each connected to a different VLAN.
  2. Unifi Network has "Multicast DNS" forwarding enabled for two (or more) of the VLANs that Home Assistant is connected to.

https://github.com/home-assistant/operating-system/issues/1163 seems to explain the reasoning behind the constantly changing hostnames:

It seems like this could happen with any multihomed system running systemd-resolved... It's probably more common with home assistant because folks try to isolate IOT endpoints.


As a workaround that would allow me to continue using NextDNS mdns discovery AND Unifi Multicast forwarding, I set up Firewall rules on my UDM pro SE that drop all incoming mdns traffic from Home Assistant network interfaces, preventing unifi from forwarding its mdns queries which caused HA/systemd-resolved to believe there was a conflict.

Since HA is already on multiple VLANs, we don't need to rely upon Unifi to forward it's mdns queries, anyway.

  1. Create a port group Multicast DNS with port 5353.
  2. Create a FW rule: LAN Local, DROP, UDP, source port group Multicast DNS, source MAC of HA interface 1.
  3. Create a FW rule: LAN Local, DROP, UDP, source port group Multicast DNS, source MAC of HA interface 2.
  4. Create a FW rule: LAN V6 Local, DROP, UDP, source port group Multicast DNS, source MAC of HA interface 1.
  5. Create a FW rule: LAN V6 Local, DROP, UDP, source port group Multicast DNS, source MAC of HA interface 2.

image

rs commented 8 months ago

Can you please test with this binary and tell me if it foxes your issue?

NEXTDNS_VERSION=pr-903/SNAPSHOT-4554031 sh -c 'sh -c "$(curl -sL https://nextdns.io/install)"'
justfalter commented 8 months ago

Can you please test with this binary and tell me if it foxes your issue?

NEXTDNS_VERSION=pr-903/SNAPSHOT-4554031 sh -c 'sh -c "$(curl -sL https://nextdns.io/install)"'

I'm still seeing the issue with that build. I've added a comment onto your PR https://github.com/nextdns/nextdns/pull/903/files#r1451796043