folbricht / routedns

DNS stub resolver, proxy and router with support for DoT, DoH, DoQ, and DTLS
BSD 3-Clause "New" or "Revised" License
476 stars 63 forks source link

Orange Pi Zero 3 - Excessive memory consumption. #406

Closed marcelloinfoweb closed 3 months ago

marcelloinfoweb commented 3 months ago

On the Orange Pi Zero 3, when running, memory consumption reaches its maximum to the point of freezing and losing connection, even though memory consumption does not decrease.

Orange Pi Zero 3 - 1G SO - Dietpi for OPZ 3

I'm currently using Blocky, as Router Dns doesn't work as expected, unfortunately.

folbricht commented 3 months ago

Can you share the config you're using when this happens?

marcelloinfoweb commented 3 months ago

Can you share the config you're using when this happens?

[bootstrap-resolver]
address = "1.1.1.1:853"
protocol = "dot"

# Listeners for the local network. Can be restricted further to specific networks
# with the "allowed-net" option
[listeners.local-udp]
address = ":53"
protocol = "udp"
resolver = "cloudflare-doh-quic-cached"

[listeners.local-tcp]
address = ":53"
protocol = "tcp"
resolver = "cloudflare-doh-quic-cached"

# Cache
#[groups.cache]
#type = "cache"
#resolvers = ["ttl-update"]
#cache-size = 8192
#cache-negative-ttl = 120
#backend = {type = "redis", redis-address = "127.0.0.1:6379", redis-key-prefix = "routedns-"}

[groups.cloudflare-doh-quic-cached]
type = "cache"
resolvers = ["cloudflare-doh-quic"]
cache-negative-ttl = 10
cache-answer-shuffle = "round-robin"
backend = {type = "redis", redis-address = "127.0.0.1:6379"}

# Update TTL to avoid noise using values that are too low
[groups.ttl-update]
type = "ttl-modifier"
resolvers = ["blocklist"]
ttl-min = 1800  # 30 Minutes
ttl-max = 43200 # 12 Hours

# Block queries (by name) using lists loaded from remote locations with HTTP and refreshed once a day
[groups.blocklist]
type = "blocklist-v2"
resolvers = ["cloudflare-doh-quic-cached"]
blocklist-refresh = 86400
blocklist-source = [
        {format = "domain", source = "https://raw.githubusercontent.com/cbuijs/accomplist/master/easylist/routedns.blocklist.domain.list", cache-dir="/tmp"},
        {format = "domain", source = "https://raw.githubusercontent.com/cbuijs/accomplist/master/malicious-dom/routedns.blocklist.domain.list", cache-dir="/tmp"},
]

# Block responses that include certain names. Also loaded via HTTP and refreshed daily
#[groups.blocklist-response]
#type = "response-blocklist-name"
#resolvers = ["blocklist-ip"]
#blocklist-refresh = 86400
#blocklist-source = [
#       {format = "domain", source = "https://raw.githubusercontent.com/cbuijs/accomplist/master/cloak/routedns.blocklist.domain.list"},
#       {format = "domain", source = "https://raw.githubusercontent.com/cbuijs/accomplist/master/malicious-dom/routedns.blocklist.domain.list"},
#]

# Block responses by IP ranges
#[groups.blocklist-ip]
#type = "response-blocklist-ip"
#resolvers = ["adguard"]
#blocklist-refresh = 86400
#blocklist-source = [
#       {format = "cidr", source = "https://raw.githubusercontent.com/cbuijs/accomplist/master/malicious-ip/plain.black.ip4cidr.list"},
#       {format = "cidr", source = "https://raw.githubusercontent.com/cbuijs/accomplist/master/malicious-ip/plain.black.ip6cidr.list"},
#]

#[groups.adguard]
#type = "fail-rotate"
#resolvers = ["adguard-doq"]

# Cloudflare DNS-over-TLS
#[resolvers.cloudflare-dot-1]
#address = "1.1.1.1:853"
#protocol = "dot"

#[resolvers.cloudflare-dot-2]
#address = "1.0.0.1:853"
#protocol = "dot"

#[resolvers.adguard-doq]
#address = "dns.nextdns.io:853"
#protocol = "doq"

[resolvers.cloudflare-doh-quic]
address = "https://1.1.1.1/dns-query"
protocol = "doh"
transport = "quic"
enable-0rtt = false
cbuijs commented 3 months ago

Hi @marcelloinfoweb,

I see you are using my lists, Cool! :-)

Tip: To cut down on memory usage due to list usage, use the "top-n" versions of the lists, these contain domains that are actually active on the internet and have been detected to be in use. I use them for years with little to no "slippage", and if I detected a mishap, the domain in question was also not in the "big" list, so should happen anyway.

The lists are updated every 24h with the latest info/trail of the last 30 days.

for example: https://raw.githubusercontent.com/cbuijs/accomplist/master/malicious-dom/routedns.blocklist.top-n.domain.list (goes from 1.5M entries to 211K entries).

You might also change the cache backend to memory instead of rediz and limit it with the size parameter. And disable rediz to free-up memory.

Hope this helps. -Chris.

marcelloinfoweb commented 3 months ago

What is the difference between using Redis or not?

I changed the configuration and updated the binary, now everything is fine.

cbuijs commented 3 months ago

Redis can be quite memory hungry and maybe a bit overkill/complex for a DNS cache (my opinion).

The memory option within RouteDNS is designed and fit for the job, so probably more efficient/quicker anyway (without testing, just assuming).