search

home-assistant / plugin-dns

CoreDNS implementation for Home Assistant
Apache License 2.0
20 stars 14 forks source link

DNS over TLS block causes high CPU / network activity #95

Closed CircuitGuy closed 2 years ago

CircuitGuy commented 2 years ago

Describe the issue you are experiencing

If port 853 (DNS over TLS) is blocked by the firewall home assistant is beating the firewall with thousands of requests per second (60k-90k per second over a period of months in my case). Specifically, it's trying to reach 1.1.1.1.853 and 1.0.0.1.853.

Aside from the very high request count, the rejection of DNS over TLS doesn't actually prevent Home Assistant from working normally. I.e. it is able to perform DNS lookups, add integrations, update itself, etc. Presumably through fallback to port 53.

Known Workarounds

Don't block port 853. :)

Expected Behavior

Rate limit on requests and fallback to port 53.

What operating system image do you use?

ova (for Virtual Machines)

What version of Home Assistant Operating System is installed?

7.6

Did you upgrade the Operating System.

No

Steps to reproduce the issue

  1. Run Home Assistant in a VM (or similar environment to show network traffic / CPU usage). Note: Home Assistant itself was not showing the high CPU usage / network traffic.
  2. Note network / CPU usage
  3. Reject port 853 in firewall.
  4. Note increase network / CPU usage
  5. Running a packet capture shows thousands of TCP requests to 1.0.0.1:853 and 1.1.1.1:853

Anything in the Supervisor logs that might be useful for us?

N/A - Supervisor logs did not indicate the issue, and show little/no CPU usage

Anything in the Host logs that might be useful for us?

DNS log does show the rejected packets.
[INFO] 127.0.0.1:34825 - 1429 "NS IN . udp 17 false 512" NOERROR - 0 5.000528289s
[ERROR] plugin/errors: 2 . NS: dial tcp 1.0.0.1:853: connect: connection refused
[INFO] 127.0.0.1:47663 - 43226 "NS IN . udp 17 false 512" NOERROR - 0 5.000066027s
[ERROR] plugin/errors: 2 . NS: dial tcp 1.1.1.1:853: connect: connection refused
[INFO] 127.0.0.1:36510 - 52909 "NS IN . udp 17 false 512" NOERROR - 0 5.001465831s
[ERROR] plugin/errors: 2 . NS: dial tcp 1.1.1.1:853: connect: connection refused
[INFO] 127.0.0.1:51123 - 22012 "NS IN . udp 17 false 512" NOERROR - 0 5.000262325s
[ERROR] plugin/errors: 2 . NS: dial tcp 1.0.0.1:853: connect: connection refused
[INFO] 127.0.0.1:37546 - 17757 "NS IN . udp 17 false 512" NOERROR - 0 5.000330775s

System Health information

System Health

version core-2021.8.8
installation_type Home Assistant OS
dev false
hassio true
docker true
user root
virtualenv false
python_version 3.9.6
os_name Linux
os_version 5.10.108
arch x86_64
timezone America/Chicago
Home Assistant Community Store GitHub API | ok -- | -- Github API Calls Remaining | 4935 Installed Version | 1.14.1 Stage | running Available Repositories | 1045 Installed Repositories | 4
Home Assistant Cloud logged_in | false -- | -- can_reach_cert_server | ok can_reach_cloud_auth | ok can_reach_cloud | ok
Home Assistant Supervisor host_os | Home Assistant OS 7.6 -- | -- update_channel | stable supervisor_version | supervisor-2022.05.3 docker_version | 20.10.9 disk_total | 17.0 GB disk_used | 12.2 GB healthy | true supported | true board | ova supervisor_api | ok version_api | ok installed_addons | Studio Code Server (4.2.0), Log Viewer (0.13.0), Node-RED (10.0.0), Samba share (9.3.1), Terminal & SSH (9.4.0), ESPHome (2021.8.2), InfluxDB (4.2.1), Grafana (7.2.0), Check Home Assistant configuration (3.10.0), Mosquitto broker (6.1.2)
Lovelace dashboards | 3 -- | -- resources | 1 views | 3 mode | storage

Additional information

No response

agners commented 2 years ago

This logs are from the DNS plugin, moving the issue accordingly.

CircuitGuy commented 2 years ago

I found the workaround: https://community.home-assistant.io/t/improve-privacy-stop-using-hardcoded-dns/273496/19

FYI - I feel like the excessive traffic / CPU elevates this from "bad best practice" and "privacy concern" to "real bug". I probably wouldn't have noticed on a RPi, but running on a more powerful VM really exacerbated this issue.

I still don't understand how DNS resolution is / was working on Home Assistant, yet it's still hammering the firewall for DNS lookups.

d0nni3q84 commented 2 years ago

This is a duplicate report of #90 which contains a root cause analysis. I haven’t heard back from the plug-in owner so I’ll begin work on revising the configuration.