Very slow and increasing "Average upstream response time"

netizeni commented 3 months ago

Prerequisites

[X] I have checked the Wiki and Discussions and found no answer
[X] I have searched other issues and found no duplicates
[X] I want to report a bug and not ask a question or ask for help
[X] I have set up AdGuard Home correctly and configured clients to use it. (Use the Discussions for help with installing and configuring clients.)

Platform (OS and CPU architecture)

Custom (please mention in the description)

Installation

Other (please mention in the description)

Setup

On one machine

AdGuard Home version

v0.107.45

Action

I used to use DoH of various DNS services and recently noticed it takes quite a while to load websites, so I decided to switch to old regular DNS, hoping to speed it up, but it didn't happen. Once added, upstream DNS starts increasing the average response time for more than 10x.

On the same machine where AGH is installed, running dnsperftest script multiple times a day, returns more or less consistent results:

                     test1   test2   test3   test4   test5   test6   test7   test8   test9   test10  Average
76.76.2.41           35 ms   35 ms   35 ms   39 ms   35 ms   39 ms   39 ms   47 ms   39 ms   35 ms     37.80  //from resolv.conf
9.9.9.9              3 ms    3 ms    47 ms   3 ms    11 ms   3 ms    7 ms    3 ms    3 ms    11 ms     9.40  //from resolv.conf
quad9                3 ms    3 ms    7 ms    3 ms    3 ms    3 ms    3 ms    3 ms    7 ms    11 ms     4.60
google               11 ms   11 ms   27 ms   11 ms   7 ms    27 ms   15 ms   59 ms   11 ms   27 ms     20.60
norton               31 ms   27 ms   31 ms   27 ms   27 ms   27 ms   27 ms   27 ms   27 ms   23 ms     27.40
neustar              31 ms   35 ms   31 ms   31 ms   35 ms   35 ms   35 ms   31 ms   31 ms   35 ms     33.00
level3               27 ms   31 ms   31 ms   63 ms   31 ms   51 ms   31 ms   31 ms   27 ms   31 ms     35.40
cleanbrowsing        35 ms   35 ms   39 ms   35 ms   35 ms   39 ms   39 ms   39 ms   35 ms   43 ms     37.40
nextdns              39 ms   39 ms   39 ms   35 ms   35 ms   39 ms   35 ms   39 ms   39 ms   39 ms     37.80
opendns              35 ms   35 ms   39 ms   39 ms   39 ms   35 ms   35 ms   51 ms   35 ms   35 ms     37.80
comodo               39 ms   35 ms   39 ms   35 ms   39 ms   35 ms   43 ms   39 ms   39 ms   39 ms     38.20
freenom              35 ms   31 ms   63 ms   31 ms   75 ms   27 ms   31 ms   83 ms   35 ms   83 ms     49.40
yandex               71 ms   67 ms   71 ms   67 ms   71 ms   71 ms   71 ms   71 ms   71 ms   67 ms     69.80
adguard              155 ms  127 ms  139 ms  175 ms  119 ms  139 ms  123 ms  155 ms  131 ms  135 ms    139.80
cloudflare           1000 ms 1000 ms 1000 ms 1000 ms 1000 ms 1000 ms 1000 ms 1000 ms 1000 ms 1000 ms   1000.00

While the current "Average upstream response time" in AGH looks like this (and progressively increases more and more):

When I'm using VPN and its DNS, website are loading noticeably faster. Is there something to change in AdGuard Home DNS settings shown below which should hopefully speed up the response time?

dns:
  bind_hosts:
    - 0.0.0.0
  port: 53
  anonymize_client_ip: false
  ratelimit: 150
  ratelimit_subnet_len_ipv4: 24
  ratelimit_subnet_len_ipv6: 56
  ratelimit_whitelist: []
  refuse_any: true
  upstream_dns:
    - 76.76.2.41
    - 76.76.2.32
    - 193.110.81.0
    - 9.9.9.9
  upstream_dns_file: ""
  bootstrap_dns:
    - 76.76.10.32
    - 76.76.10.41
  fallback_dns:
    - 9.9.9.9
  upstream_mode: load_balance
  fastest_timeout: 1s
  allowed_clients: []
  disallowed_clients: []
  blocked_hosts:
    - version.bind
    - id.server
    - hostname.bind
  trusted_proxies:
    - 127.0.0.0/8
    - ::1/128
  cache_size: 134217728
  cache_ttl_min: 0
  cache_ttl_max: 0
  cache_optimistic: true
  bogus_nxdomain: []
  aaaa_disabled: false
  enable_dnssec: false
  edns_client_subnet:
    custom_ip: ""
    enabled: false
    use_custom: false
  max_goroutines: 300
  handle_ddr: true
  ipset: []
  ipset_file: ""
  bootstrap_prefer_ipv6: false
  upstream_timeout: 10s
  private_networks: []
  use_private_ptr_resolvers: true
  local_ptr_upstreams: []
  use_dns64: false
  dns64_prefixes: []
  serve_http3: false
  use_http3_upstreams: false
  serve_plain_dns: true

Expected result

Lower "Average upstream response time" over time and faster responses.

Actual result

"Average upstream response time" getting increased over time. Websites take quite a while to load.

Additional information and/or screenshots

AdGuard Home is installed on RPi 3B+ running DietPi (debian based).

bobloadmire commented 3 months ago

I have this exact same issue, especially with cloudflare. I had to remove 1.1.1.1 completely, but i'm still having issues using only google and opendns. running on rpi4

whyisthisbroken commented 3 months ago

i use unbound and dnscrypt proxy upstream to quad9 resolver. but maybe you can test following changes:

ratelimit: 0
refuse_any: false
upstream_mode: parallel
fastest_timeout: 1s
enable_dnssec: true
max_goroutines: 500
handle_ddr: true
upstream_timeout: 2s

Optional:
bootstrap_prefer_ipv6: true

I don't know ControlD and its service - but maybe you could test it without it. Your ControlD settings use OISD and Hagezi's blacklist - do you also have these lists in AGH? The third DNS service is dns0 – the nextdns eu brother, also with filter lists...

It's all doubled twice. - Its not bad, but for testing we should use only one DNS Service to exclude the problem.... Which filter lists do you use in AGH?

My results for the DNS Service you use:

root@HomeNetDNS:~# mtr -r -w -c4 193.110.81.0
Start: 2024-03-16T08:53:35+0100
HOST: HomeNetDNS                                Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- Fritzbox                                   0.0%     4    0.7   0.8   0.6   0.9   0.1
  2.|-- 100.124.1.76                               0.0%     4    8.8   8.7   6.4  12.0   2.4
  3.|-- 100.127.1.133                              0.0%     4    6.8   6.7   5.6   8.5   1.3
  4.|-- 100.127.1.132                              0.0%     4    6.2   6.1   4.4   7.7   1.3
  5.|-- 185.22.46.129                              0.0%     4    5.9  20.9   5.9  37.3  16.9
  6.|-- ae3-1337.bbr02.anx63.ams.nl.anexia-it.net  0.0%     4   18.4  17.4  15.8  19.4   1.8
  7.|-- ae1-10.bbr01.anx63.ams.nl.anexia-it.net    0.0%     4   16.3  16.1  15.0  17.7   1.2
  8.|-- dns0.eu                                    0.0%     4   14.4  15.6  14.4  17.3   1.3
root@HomeNetDNS:~# mtr -r -w -c4 76.76.2.41
Start: 2024-03-16T08:55:00+0100
HOST: HomeNetDNS    Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- Fritzbox       0.0%     4    0.7   0.8   0.7   0.9   0.1
  2.|-- 100.124.1.76   0.0%     4    6.8   7.2   5.3   8.5   1.5
  3.|-- 100.127.1.132  0.0%     4    6.4   7.0   5.5   8.4   1.3
  4.|-- 185.22.46.129  0.0%     4    8.3   8.1   6.4   9.2   1.2
  5.|-- ???           100.0     4    0.0   0.0   0.0   0.0   0.0
  6.|-- ???           100.0     4    0.0   0.0   0.0   0.0   0.0
  7.|-- 76.76.2.41     0.0%     4   12.1  12.5  11.1  13.6   1.1
root@HomeNetDNS:~# mtr -r -w -c4 76.76.2.32
Start: 2024-03-16T08:55:18+0100
HOST: HomeNetDNS    Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- Fritzbox       0.0%     4    0.9   0.9   0.8   1.2   0.2
  2.|-- 100.124.1.76   0.0%     4    7.6   8.5   7.0  10.3   1.5
  3.|-- 100.127.1.131  0.0%     4    5.5   6.7   4.8   9.4   2.0
  4.|-- 100.127.1.132  0.0%     4    4.6   6.8   4.6   9.2   2.0
  5.|-- 185.22.46.145  0.0%     4    8.2   7.9   6.3   9.7   1.4
  6.|-- ???           100.0     4    0.0   0.0   0.0   0.0   0.0
  7.|-- ???           100.0     4    0.0   0.0   0.0   0.0   0.0
  8.|-- 76.76.2.32     0.0%     4   11.6  10.9   9.1  12.7   1.6
root@HomeNetDNS:~# mtr -r -w -c4 9.9.9.9
Start: 2024-03-16T08:56:15+0100
HOST: HomeNetDNS                   Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- Fritzbox                      0.0%     4    1.1   0.9   0.6   1.1   0.3
  2.|-- 100.124.1.76                  0.0%     4    5.9   7.1   5.9   8.5   1.1
  3.|-- 100.127.1.133                 0.0%     4    5.4   5.8   4.3   7.1   1.2
  4.|-- 100.127.1.132                 0.0%     4    6.7   6.7   5.8   7.8   0.9
  5.|-- 185.22.46.145                 0.0%     4    8.4   7.2   5.7   8.4   1.1
  6.|-- as42.dusseldorf.megaport.com  0.0%     4    9.1  10.4   9.1  12.0   1.2
  7.|-- dns9.quad9.net                0.0%     4    9.5  11.0   9.5  12.4   1.2

netizeni commented 3 months ago

@whyisthisbroken I will check those settings, thanks. Do you use quad9 DoH and should upstream_mode: parallel speed it up a bit comparing to load balance?

Your ControlD settings use OISD and Hagezi's blacklist - do you also have these lists in AGH? The third DNS service is dns0 – the nextdns eu brother, also with filter lists...

Yes, I do. Even though it might be redundant, I tested these upstream DNS with and without filter lists and the processing time was the same, so I decided to use those with filter lists anyway.

After setting blocking_mode: null_ip and leaving ControlD and Quad9 for a couple of days (removed dns0.eu), seems like the situation is a bit better.

Now in General statistics "Average processing time" is 8ms, although "Average upstream response time" is still quite big

76.76.2.41:53   220 ms
76.76.2.32:53   217 ms
9.9.9.9:53      196 ms

I assume this means the most of DNS queries are served from cache, hence why this 8ms, while, when necessary, querying upstream gives these big numbers above?

Also, one semi-related question. One client has always-on VPN and all the traffic goes through it, meanwhile AGH logs every minute or so a query from that client to www.google.com Type: A, Plain DNS. How is that possible?

dMopp commented 3 months ago

Can confirm the issue. Have to use my local root resolver. 1.1.1.1 (and ipv6) are terrible slow from adguard.

Morlince commented 3 months ago

I think I'm having the same issue. It's improved with some of the suggestions here, but something still seems off.

dansseg commented 3 months ago

Same issue here. Restarting the docker container helps for couple of hours...

netizeni commented 3 months ago

It's definitely abnormal and constantly increasing. The latest results for "Average upstream response time":

76.76.2.41:53   1021 ms
76.76.2.32:53   1006 ms
9.9.9.9:53      750 ms

Cebeerre commented 3 months ago

It's definitely abnormal and constantly increasing. The latest results for "Average upstream response time":
76.76.2.41:53   1021 ms
76.76.2.32:53   1006 ms
9.9.9.9:53      750 ms

Where are you located ? Keep in mind that even though ControlD or Quad9 are both using unicast, the peerings used by your ISP might affect quite a lot ...

netizeni commented 3 months ago

Where are you located ?

I'm in Europe and there are multiple quad9 server locations near me, which dnsperftest script confirms as it shows 3ms on average.

Unfortunately, with AGH that's not the case considering from my last reply "Average upstream response time" now increased to:

76.76.2.32:53   5856 ms
9.9.9.9:53      5320 ms
76.76.2.41:53   4763 ms

Cebeerre commented 3 months ago

You maybe already did, but could you run an extended test on https://www.dnsleaktest.com/ ? to check which locations are actually answering your requests ?

netizeni commented 3 months ago

Sure.

176.58.88.155  |  lhr-h05.int.controld.com.  |  NetActuate
176.58.88.250  |  lhr-h04.int.controld.com.  |  NetActuate
66.185.117.242  |  res100.vie.rrdns.pch.net.  |  WoodyNet
66.185.117.243  |  res200.vie.rrdns.pch.net.  |  WoodyNet
66.185.117.244  |  res300.vie.rrdns.pch.net.  |  WoodyNet

Based on the domain of last three, seems like they are in the same city. One more thing I noticed, in the past couple of days average upstream response time decreased, but only slightly, as numbers are still huge:

76.76.2.32:53   5145 ms
9.9.9.9:53      4596 ms
76.76.2.41:53   3775 ms

Meanwhile, "Average processing time" increased from 8-9ms to now 153ms.

dMopp commented 3 months ago

I have the issue with my ISP DNS as well… Using my unbound (and let using it my isp dns) didn’t have these issue. So it’s a adguard thing since some versions.

Akira46 commented 3 months ago

I also got the same problem

netizeni commented 3 months ago

@Cebeerre would you mind removing "waiting for data" label and add "bug" label, please? All these replies beside mine imply it's definitely a bug.

Cebeerre commented 3 months ago

Hi @netizeni

In order tag an issue as a bug, reproducibility is key. The previous replies, apart from saying "me too", are not actually adding any additional information that might help.

This is my own AGH instance using the NS you provided after running for some hours:

Have you tried to clear the statistics and see if maybe it was just a very bad connectivity period (AGH timesouts by default at 10ms) ?

dMopp commented 3 months ago

As mentioned, I can add the exact same DNS servers to my opnsense instead of agh and have no issue. It doesn’t matter if i use my isp dns or Cloudflare.

Cebeerre commented 3 months ago

Have you tried to see if there are specific queries that are making the average increase ?

cat querylog.json | jq -r '(.QH + ":" + (.Elapsed | tostring))' | sort -t: -nrk2 | head -20

netizeni commented 3 months ago

@Cebeerre I did a couple of times ever since opening this issue, and it starts with those numbers you shared, but eventually increases to these I pasted.

Cebeerre commented 3 months ago

@Cebeerre I did a couple of times ever since opening this issue, and it starts with those numbers you shared, but eventually increases to these I pasted.

I've seen that you've set a cache size of 134 Mb which honestly looks quite overkill ... Could you please check how much RAM you're actually consuming right now by the AGH process ? Do you have other stuff running on top of this rpi3 ?

dMopp commented 3 months ago

cat querylog.json | jq -r '(.QH + ":" + (.Elapsed | tostring))' | sort -t: -nrk2 | head -20

tostring))' | sort -t: -nrk2 | head -20 login.aliexpress.com:10027762671 a1931.dscgi3.akamai.net:10024301298 cdn.smoot.g.aaplimg.com:10024288682 a1931.dscgi3.akamai.net:10021771928 user.17track.net:840082388 35ne6z.tdum.alibaba.com:777572896 35ne6z.tdum.alibaba.com:566845144 www.17track.net:460536210 www.17track.net:456030279 res.17track.net:439784586 t.17track.net:423843345 res.17track.net:422288740 s.17track.net:417543098 t.17track.net:414760661 t.17track.net:407802627 s.17track.net:403705146 res.17track.net:401303256 video-cdn.aliexpress-media.com.queniuak.com:383318976 wvcfg.alicdn.com.danuoyi.tbcache.com:363343607 h5.m.taobao.com:294407903

but currently the numbers are ok with my isp dns… in will try 1.1.1.1

Cebeerre commented 3 months ago

login.aliexpress.com:10027762671 a1931.dscgi3.akamai.net:10024301298 cdn.smoot.g.aaplimg.com:10024288682 a1931.dscgi3.akamai.net:10021771928

Wow ! All of these ones are over the 10 seconds ! quite odd given that you're using a public resolver and these entries were probably in their cache already ...

I use unbound as a recursive DNS upstream, which tipically "takes more time" and this is what I get for login.aliexpress.com:

@dMopp , what kind of hardware are you using ? Am I right assuming it's directly connected by ethernet and you're not using wifi ?

dMopp commented 3 months ago

login.aliexpress.com:10027762671 a1931.dscgi3.akamai.net:10024301298 cdn.smoot.g.aaplimg.com:10024288682 a1931.dscgi3.akamai.net:10021771928

Wow ! All of these ones are over the 10 seconds ! quite odd given that you're using a public resolver and these entries were probably in their cache already ...

I use unbound as a recursive DNS upstream, which tipically "takes more time" and this is what I get for login.aliexpress.com:

@dMopp , what kind of hardware are you using ? Am I right assuming it's directly connected by ethernet and you're not using wifi ?

Hi.

Yes this is extreme. As mentioned, using the same resolver in opnsense (unbound) I don’t see this spikes. I can even add my opnsense as dns for adguard home and it’s fine.

The Hardware: 5700x + 128GB RAM + Intel NIC.

Adguard Home and opnsense are both running in proxmox.

Ah and yes, all wired. ICMP requests are not spiking.

Cebeerre commented 3 months ago

Adguard Home and opnsense are both running in proxmox.

lxc or vm ? Have you tried to install the AdGuardHome plugin in proxmox itself to see if it makes any difference ?

dMopp commented 3 months ago

Opnsense running as a VM with PCIe pass through . AGH as LXC in a Debian container. (And yes, in the past this was fine). And which plugin you mean?

Cebeerre commented 3 months ago

And which plugin you mean?

https://www.routerperformance.net/opnsense-repo/

dMopp commented 3 months ago

Ah, you mean in OPNsense. No, because I don’t want the filtering in the firewall. This would cause some new issues.

Cebeerre commented 3 months ago

Ah, you mean in OPNsense. No, because I don’t want the filtering in the firewall. This would cause some new issues.

I'm curious about when you said that if you set the unbound instance in OPNSense as an AGH upstream it works fine. It shouldn't make any kind of difference than using a public resolver, what made me think if you've any kind of traffic shaping rules applied in OPNSense and the LXC container fell apart in an upload or download pipe without enough bandwidth ?

dMopp commented 3 months ago

Indeed i have traffic shaping, but just 2 Pipes and every traffic is passing them. UDP / 53 Traffic is even high prio in my network (independent from the Source/Destination)

Cebeerre commented 3 months ago

Indeed i have traffic shaping, but just 2 Pipes and every traffic is passing thum. UDP / 53 Traffic is even high prio in my network (independent from the Source/Destination)

Great, could you please share your entire adguardhome.yaml ?

dMopp commented 3 months ago

http:
  pprof:
    port: 6060
    enabled: false
  address: 0.0.0.0:3001
  session_ttl: 720h
users:
<snip>
auth_attempts: 5
block_auth_min: 15
http_proxy: ""
language: en
theme: auto
dns:
  bind_hosts:
    - 192.168.10.10
    - fd11:192:168:10::10
  port: 53
  anonymize_client_ip: false
  ratelimit: 0
  ratelimit_subnet_len_ipv4: 24
  ratelimit_subnet_len_ipv6: 64
  ratelimit_whitelist: []
  refuse_any: false
  upstream_dns:
    - 192.168.1.1
    - fd11:192:168:1::1
    - '#[/*.publicisgroupe.net/]81.200.178.'
    - '### Telekom'
    - '#217.237.151.51'
    - '#217.237.149.205'
    - '#2003:180:2::53'
    - '#2003:180:2:6000::53'
    - '### Cloudflare'
    - '#1.1.1.1'
    - '#1.0.0.1'
    - '#2606:4700:4700::1111'
    - '#2606:4700:4700::1001'
    - '#h3://1dot1dot1dot1.cloudflare-dns.com/dns-query'
    - '### Router'
    - '#[/*.dmopp.de/]192.168.1.1'
    - '#[/*.dmopp.de/]fd11:192:168:1::1'
  upstream_dns_file: ""
  bootstrap_dns:
    - 192.168.1.1
    - fd11:192:168:1::1
  fallback_dns:
    - 192.168.1.1
    - fd11:192:168:1::1
  upstream_mode: load_balance
  fastest_timeout: 1s
  allowed_clients: []
  disallowed_clients: []
  blocked_hosts:
    - version.bind
    - id.server
    - hostname.bind
    - wpad.servers.dmopp.de
    - '*.home.arpa'
  trusted_proxies:
    - 127.0.0.0/8
    - ::1/128
    - 192.168.1.55/32
    - fd11:192:168:1::55/128
  cache_size: 10000
  cache_ttl_min: 60
  cache_ttl_max: 0
  cache_optimistic: true
  bogus_nxdomain: []
  aaaa_disabled: false
  enable_dnssec: true
  edns_client_subnet:
    custom_ip: 91.15.194.225
    enabled: false
    use_custom: false
  max_goroutines: 300
  handle_ddr: true
  ipset: []
  ipset_file: ""
  bootstrap_prefer_ipv6: false
  upstream_timeout: 10s
  private_networks: []
  use_private_ptr_resolvers: true
  local_ptr_upstreams:
    - 192.168.1.1
    - fd11:192:168:1::1
  use_dns64: false
  dns64_prefixes: []
  serve_http3: false
  use_http3_upstreams: false
  serve_plain_dns: true
  hostsfile_enabled: true
tls:
  enabled: true
  server_name: dns.lan.dmopp.de
  force_https: false
  port_https: 443
  port_dns_over_tls: 853
  port_dns_over_quic: 853
  port_dnscrypt: 0
  dnscrypt_config_file: ""
  allow_unencrypted_doh: false
  certificate_chain: ""
  private_key: ""
  certificate_path: /root/.acme.sh/dns.lan.dmopp.de_ecc/fullchain.cer
  private_key_path: /root/.acme.sh/dns.lan.dmopp.de_ecc/dns.lan.dmopp.de.key
  strict_sni_check: false
querylog:
  dir_path: ""
  ignored:
    - /.*dns-sd._udp.*/
    - /.*ldap._tcp.dc.*/
    - /.*.dmopp.de/
    - /.*.thread.home.arpa/
  interval: 48h
  size_memory: 1000
  enabled: true
  file_enabled: true
statistics:
  dir_path: ""
  ignored:
    - /.*dns-sd._udp.*/
    - /.*ldap._tcp.dc.*/
    - /.*.dmopp.de/
    - /.*.thread.home.arpa/
  interval: 48h
  enabled: true
filters:
  - enabled: true
    url: https://raw.githubusercontent.com/PolishFiltersTeam/KADhosts/master/KADhosts.txt
    name: https://raw.githubusercontent.com/PolishFiltersTeam/KADhosts/master/KADhosts.txt
    id: 1697381928
  - enabled: true
    url: https://raw.githubusercontent.com/FadeMind/hosts.extras/master/add.Spam/hosts
    name: https://raw.githubusercontent.com/FadeMind/hosts.extras/master/add.Spam/hosts
    id: 1697381929
  - enabled: true
    url: https://v.firebog.net/hosts/static/w3kbl.txt
    name: https://v.firebog.net/hosts/static/w3kbl.txt
    id: 1697381930
  - enabled: true
    url: https://raw.githubusercontent.com/matomo-org/referrer-spam-blacklist/master/spammers.txt
    name: https://raw.githubusercontent.com/matomo-org/referrer-spam-blacklist/master/spammers.txt
    id: 1697381931
  - enabled: true
    url: https://someonewhocares.org/hosts/zero/hosts
    name: https://someonewhocares.org/hosts/zero/hosts
    id: 1697381932
  - enabled: true
    url: https://raw.githubusercontent.com/VeleSila/yhosts/master/hosts
    name: https://raw.githubusercontent.com/VeleSila/yhosts/master/hosts
    id: 1697381933
  - enabled: true
    url: https://winhelp2002.mvps.org/hosts.txt
    name: https://winhelp2002.mvps.org/hosts.txt
    id: 1697381934
  - enabled: true
    url: https://v.firebog.net/hosts/neohostsbasic.txt
    name: https://v.firebog.net/hosts/neohostsbasic.txt
    id: 1697381935
  - enabled: true
    url: https://raw.githubusercontent.com/RooneyMcNibNug/pihole-stuff/master/SNAFU.txt
    name: https://raw.githubusercontent.com/RooneyMcNibNug/pihole-stuff/master/SNAFU.txt
    id: 1697381936
  - enabled: true
    url: https://paulgb.github.io/BarbBlock/blacklists/hosts-file.txt
    name: https://paulgb.github.io/BarbBlock/blacklists/hosts-file.txt
    id: 1697381937
  - enabled: true
    url: https://adaway.org/hosts.txt
    name: https://adaway.org/hosts.txt
    id: 1697381938
  - enabled: true
    url: https://v.firebog.net/hosts/AdguardDNS.txt
    name: https://v.firebog.net/hosts/AdguardDNS.txt
    id: 1697381939
  - enabled: true
    url: https://v.firebog.net/hosts/Admiral.txt
    name: https://v.firebog.net/hosts/Admiral.txt
    id: 1697381940
  - enabled: true
    url: https://raw.githubusercontent.com/anudeepND/blacklist/master/adservers.txt
    name: https://raw.githubusercontent.com/anudeepND/blacklist/master/adservers.txt
    id: 1697381941
  - enabled: true
    url: https://s3.amazonaws.com/lists.disconnect.me/simple_ad.txt
    name: https://s3.amazonaws.com/lists.disconnect.me/simple_ad.txt
    id: 1697381942
  - enabled: true
    url: https://v.firebog.net/hosts/Easylist.txt
    name: https://v.firebog.net/hosts/Easylist.txt
    id: 1697381943
  - enabled: true
    url: https://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&showintro=0&mimetype=plaintext
    name: https://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&showintro=0&mimetype=plaintext
    id: 1697381944
  - enabled: true
    url: https://raw.githubusercontent.com/FadeMind/hosts.extras/master/UncheckyAds/hosts
    name: https://raw.githubusercontent.com/FadeMind/hosts.extras/master/UncheckyAds/hosts
    id: 1697381945
  - enabled: true
    url: https://raw.githubusercontent.com/bigdargon/hostsVN/master/hosts
    name: https://raw.githubusercontent.com/bigdargon/hostsVN/master/hosts
    id: 1697381946
  - enabled: true
    url: https://raw.githubusercontent.com/jdlingyu/ad-wars/master/hosts
    name: https://raw.githubusercontent.com/jdlingyu/ad-wars/master/hosts
    id: 1697381947
  - enabled: true
    url: https://v.firebog.net/hosts/Easyprivacy.txt
    name: https://v.firebog.net/hosts/Easyprivacy.txt
    id: 1697381948
  - enabled: true
    url: https://v.firebog.net/hosts/Prigent-Ads.txt
    name: https://v.firebog.net/hosts/Prigent-Ads.txt
    id: 1697381949
  - enabled: true
    url: https://raw.githubusercontent.com/FadeMind/hosts.extras/master/add.2o7Net/hosts
    name: https://raw.githubusercontent.com/FadeMind/hosts.extras/master/add.2o7Net/hosts
    id: 1697381950
  - enabled: true
    url: https://raw.githubusercontent.com/crazy-max/WindowsSpyBlocker/master/data/hosts/spy.txt
    name: https://raw.githubusercontent.com/crazy-max/WindowsSpyBlocker/master/data/hosts/spy.txt
    id: 1697381951
  - enabled: true
    url: https://hostfiles.frogeye.fr/firstparty-trackers-hosts.txt
    name: https://hostfiles.frogeye.fr/firstparty-trackers-hosts.txt
    id: 1697381952
  - enabled: true
    url: https://www.github.developerdan.com/hosts/lists/ads-and-tracking-extended.txt
    name: https://www.github.developerdan.com/hosts/lists/ads-and-tracking-extended.txt
    id: 1697381953
  - enabled: true
    url: https://raw.githubusercontent.com/Perflyst/PiHoleBlocklist/master/android-tracking.txt
    name: https://raw.githubusercontent.com/Perflyst/PiHoleBlocklist/master/android-tracking.txt
    id: 1697381954
  - enabled: true
    url: https://raw.githubusercontent.com/Perflyst/PiHoleBlocklist/master/SmartTV.txt
    name: https://raw.githubusercontent.com/Perflyst/PiHoleBlocklist/master/SmartTV.txt
    id: 1697381955
  - enabled: true
    url: https://raw.githubusercontent.com/Perflyst/PiHoleBlocklist/master/AmazonFireTV.txt
    name: https://raw.githubusercontent.com/Perflyst/PiHoleBlocklist/master/AmazonFireTV.txt
    id: 1697381956
  - enabled: true
    url: https://gitlab.com/quidsup/notrack-blocklists/raw/master/notrack-blocklist.txt
    name: https://gitlab.com/quidsup/notrack-blocklists/raw/master/notrack-blocklist.txt
    id: 1697381957
  - enabled: true
    url: https://raw.githubusercontent.com/DandelionSprout/adfilt/master/Alternate%20versions%20Anti-Malware%20List/AntiMalwareHosts.txt
    name: https://raw.githubusercontent.com/DandelionSprout/adfilt/master/Alternate%20versions%20Anti-Malware%20List/AntiMalwareHosts.txt
    id: 1697381958
  - enabled: true
    url: https://osint.digitalside.it/Threat-Intel/lists/latestdomains.txt
    name: https://osint.digitalside.it/Threat-Intel/lists/latestdomains.txt
    id: 1697381959
  - enabled: true
    url: https://s3.amazonaws.com/lists.disconnect.me/simple_malvertising.txt
    name: https://s3.amazonaws.com/lists.disconnect.me/simple_malvertising.txt
    id: 1697381960
  - enabled: true
    url: https://v.firebog.net/hosts/Prigent-Crypto.txt
    name: https://v.firebog.net/hosts/Prigent-Crypto.txt
    id: 1697381961
  - enabled: true
    url: https://raw.githubusercontent.com/FadeMind/hosts.extras/master/add.Risk/hosts
    name: https://raw.githubusercontent.com/FadeMind/hosts.extras/master/add.Risk/hosts
    id: 1697381962
  - enabled: true
    url: https://bitbucket.org/ethanr/dns-blacklists/raw/8575c9f96e5b4a1308f2f12394abd86d0927a4a0/bad_lists/Mandiant_APT1_Report_Appendix_D.txt
    name: https://bitbucket.org/ethanr/dns-blacklists/raw/8575c9f96e5b4a1308f2f12394abd86d0927a4a0/bad_lists/Mandiant_APT1_Report_Appendix_D.txt
    id: 1697381963
  - enabled: true
    url: https://phishing.army/download/phishing_army_blocklist_extended.txt
    name: https://phishing.army/download/phishing_army_blocklist_extended.txt
    id: 1697381964
  - enabled: true
    url: https://gitlab.com/quidsup/notrack-blocklists/raw/master/notrack-malware.txt
    name: https://gitlab.com/quidsup/notrack-blocklists/raw/master/notrack-malware.txt
    id: 1697381965
  - enabled: true
    url: https://v.firebog.net/hosts/RPiList-Malware.txt
    name: https://v.firebog.net/hosts/RPiList-Malware.txt
    id: 1697381966
  - enabled: true
    url: https://v.firebog.net/hosts/RPiList-Phishing.txt
    name: https://v.firebog.net/hosts/RPiList-Phishing.txt
    id: 1697381967
  - enabled: true
    url: https://raw.githubusercontent.com/Spam404/lists/master/main-blacklist.txt
    name: https://raw.githubusercontent.com/Spam404/lists/master/main-blacklist.txt
    id: 1697381968
  - enabled: true
    url: https://raw.githubusercontent.com/AssoEchap/stalkerware-indicators/master/generated/hosts
    name: https://raw.githubusercontent.com/AssoEchap/stalkerware-indicators/master/generated/hosts
    id: 1697381969
  - enabled: true
    url: https://urlhaus.abuse.ch/downloads/hostfile/
    name: https://urlhaus.abuse.ch/downloads/hostfile/
    id: 1697381970
  - enabled: true
    url: https://malware-filter.gitlab.io/malware-filter/phishing-filter-hosts.txt
    name: https://malware-filter.gitlab.io/malware-filter/phishing-filter-hosts.txt
    id: 1697381971
  - enabled: true
    url: https://v.firebog.net/hosts/Prigent-Malware.txt
    name: https://v.firebog.net/hosts/Prigent-Malware.txt
    id: 1697381972
  - enabled: true
    url: https://zerodot1.gitlab.io/CoinBlockerLists/hosts_browser
    name: https://zerodot1.gitlab.io/CoinBlockerLists/hosts_browser
    id: 1697381973
whitelist_filters:
  - enabled: true
    url: https://raw.githubusercontent.com/anudeepND/whitelist/master/domains/whitelist.txt
    name: https://raw.githubusercontent.com/anudeepND/whitelist/master/domains/whitelist.txt
    id: 1692541159
user_rules:
  - '##Whitelist'
  - '@@||cdn.tinypass.com^$important'
  - '@@||c2-eu.piano.io^$important'
  - '@@||buy-eu.piano.io^$important'
  - '@@||gs-loc.apple.com^$important'
  - '@@||local^$important'
  - '@@||url2478.gigabyte.com^$important'
  - '##Blacklist'
  - '||dns.google^$important'
  - '||bet365.odds.am^$important'
  - '||bcgame.sptpub.com^$important'
  - '||bc.fun^$important'
  - '##Regex Blacklist'
  - /.*dns-sd._udp.*/$important
  - '##Regex Whitelist'
  - '||www.ad-production-stage.com^$important'
  - '@@||logging.dhg.myharmony.com^$client=''192.168.99.200'''
  - '@@||logging.dhg.myharmony.com^$important'
  - '@@||cstat.cdn-apple.com^$important'
  - '@@||www.googleadservices.com^$important'
  - '||7ed655f2dae0a0e6b2a852d5a33396183a35ec681d56c16df8c02b65d01a481.us-east-1.prod.service.minerva.devices.a2z.com.iot.dmopp.de^$important'
  - '||www.autodoc.de^$important'
  - '||m.autodoc.de^$important'
  - '@@||assets.adobedtm.com^$client=''192.168.4.194'''
  - '@@||tags.tiqcdn.com^$client=''192.168.4.194'''
  - '@@||conduit.redfast.com^$client=''192.168.4.194'''
  - '@@||cydia.saurik.com^$important'
  - '@@||mask.icloud.com^$important'
  - '@@||feedbackws.icloud.com^$important'
  - '@@||meinkonto.telekom-dienste.de^$important'
  - '@@||is1-ssl.mzstatic.com^$important'
  - '@@||image.ard.de^$important'
  - '@@||image-ard-de-cddc.at-o.net^$important'
  - '@@||api.smoot.apple.com^$important'
  - '@@||s.click.aliexpress.com^$important'
  - '||nrdp.prod.ftl.netflix.com^$client=''192.168.8.100'''
  - '||api-global.netflix.com^$client=''192.168.8.100'''
  - '||cdn-0.nflximg.com^$client=''192.168.99.134'''
  - '||appboot.netflix.com^$client=''192.168.8.100'''
  - '||cdn-0.nflximg.com^$client=''192.168.8.100'''
  - '||uiboot.netflix.com^$client=''192.168.99.134'''
  - '||api-global.netflix.com^$client=''192.168.99.134'''
  - '||nrdp51-appboot.netflix.com^$client=''192.168.99.134'''
  - '||nrdp51-appboot.netflix.com^$client=''192.168.8.100'''
  - '||secure.netflix.com^$client=''192.168.99.134'''
  - '||nrdp.prod.cloud.netflix.com^$client=''192.168.99.134'''
  - '||www.youtube.com^$client=''192.168.99.134'''
  - '||unagi-eu.amazon.com^$client=''192.168.99.134'''
  - '||piv-ignx-aehboccg7glwh-0.eu.api.amazonvideo.com^$client=''192.168.99.134'''
  - ""
dhcp:
  enabled: false
  interface_name: ""
  local_domain_name: lan
  dhcpv4:
    gateway_ip: ""
    subnet_mask: ""
    range_start: ""
    range_end: ""
    lease_duration: 86400
    icmp_timeout_msec: 1000
    options: []
  dhcpv6:
    range_start: ""
    lease_duration: 86400
    ra_slaac_only: false
    ra_allow_slaac: false
filtering:
  blocking_ipv4: ""
  blocking_ipv6: ""
  blocked_services:
    schedule:
      time_zone: Local
    ids:
      - facebook
      - icloud_private_relay
  protection_disabled_until: null
  safe_search:
    enabled: false
    bing: true
    duckduckgo: true
    google: true
    pixabay: true
    yandex: true
    youtube: true
  blocking_mode: null_ip
  parental_block_host: family-block.dns.adguard.com
  safebrowsing_block_host: standard-block.dns.adguard.com
  rewrites:
    - domain: opnsense.lan.dmopp.de
      answer: 192.168.1.1
    - domain: opnsense.lan.dmopp.de
      answer: fd11:192:168:1::1
    - domain: dns.msftncsi.com
      answer: fd3e:4f5a:5b81::1
    - domain: dns.msftncsi.com
      answer: 131.107.255.255
  safebrowsing_cache_size: 1048576
  safesearch_cache_size: 1048576
  parental_cache_size: 1048576
  cache_time: 30
  filters_update_interval: 72
  blocked_response_ttl: 360
  filtering_enabled: true
  parental_enabled: false
  safebrowsing_enabled: false
  protection_enabled: true
clients:
  runtime_sources:
    whois: true
    arp: true
    rdns: true
    dhcp: true
    hosts: true
  persistent:
    - safe_search:
        enabled: false
        bing: true
        duckduckgo: true
        google: true
        pixabay: true
        yandex: true
        youtube: true
      blocked_services:
        schedule:
          time_zone: Local
        ids: []
      name: Homeassistant
      ids:
        - 192.168.10.55
        - fd11:192:168:10::55
      tags: []
      upstreams: []
      uid: 018d7fc3-5391-7cec-9e23-d1619844b01d
      upstreams_cache_size: 0
      upstreams_cache_enabled: false
      use_global_settings: true
      filtering_enabled: false
      parental_enabled: false
      safebrowsing_enabled: false
      use_global_blocked_services: true
      ignore_querylog: false
      ignore_statistics: true
    - safe_search:
        enabled: false
        bing: true
        duckduckgo: true
        google: true
        pixabay: true
        yandex: true
        youtube: true
      blocked_services:
        schedule:
          time_zone: Local
        ids: []
      name: dMopp-PC
      ids:
        - fd11:192:168:4:44:c359:e068:ac83
      tags: []
      upstreams: []
      uid: 018e1f95-7bfe-7eaf-9917-918bfeb69c18
      upstreams_cache_size: 0
      upstreams_cache_enabled: false
      use_global_settings: true
      filtering_enabled: false
      parental_enabled: false
      safebrowsing_enabled: false
      use_global_blocked_services: true
      ignore_querylog: false
      ignore_statistics: false
log:
  file: syslog
  max_backups: 0
  max_size: 100
  max_age: 3
  compress: false
  local_time: false
  verbose: false
os:
  group: ""
  user: ""
  rlimit_nofile: 0
schema_version: 28

Cebeerre commented 3 months ago

cache_size: 10000

10000 bytes of cache ? That's quite below the default 4 Mb ... any reason for this ? Given that a DNS response might be around 50 or 100 bytes ... you can maybe hold between 100 and 200 entries in the cache ?

dMopp commented 3 months ago

Yes and no. The idea is, that only the most frequent are cached in agh and everything else in unbound (as long as iam using the unbound behind). With external resolvers, i will add two more zeros behind the number :D

Cebeerre commented 3 months ago

@dMopp @netizeni I'd suggest to you both to set up the statistics retention for the last 24h, configure your desired upstream DNS, set a cache size to something reasonable between 4 and 8 mb, optimistic cache (so AGH prefetches like unbound) and let it go for a day or two and report back.

netizeni commented 3 months ago

@Cebeerre I did a couple of times ever since opening this issue, and it starts with those numbers you shared, but eventually increases to these I pasted.

I've seen that you've set a cache size of 134 Mb which honestly looks quite overkill ... Could you please check how much RAM you're actually consuming right now by the AGH process ? Do you have other stuff running on top of this rpi3 ?

It was a default value, so I increased in an effort to fix this issue. AGH process takes 191MB and I do have other stuff running beside AGH, but even then, RAM is below 50% usage, while CPU is around 2%.

dMopp commented 3 months ago

Will wait a bit now.

dMopp commented 3 months ago

Already rising and peaks are visible in the log:


app-prod-ws.warnwetter.de:10066663857
oauth2.googleapis.com:10063205612
p27-content.icloud.com:10061289899
profile.xboxlive.com:10059952066
gsp57-ssl-revgeo.ls.apple.com:10057142519
gew4-spclient.spotify.com:10055349931
p27-content.icloud.com:10052925917
p59-content.icloud.com:10046673123
oauth2.googleapis.com:10044561643
oauth2.googleapis.com:10044256689
event.mshopbugsnag.irm.amazon.dev:10038882742
mesu.g.aaplimg.com:10037358457
displaycatalog.mp.microsoft.com:10028921868
gspe1-ssl.ls.apple.com.edgesuite.net:10028164963
weu.core.gssv-play-prod.xboxlive.com:10022560938
weu.core.gssv-play-prod.xboxlive.com:10021844701
e6858.dscx.akamaiedge.net:10020623169
msh.amazon.co.uk:10020456046
raw.githubusercontent.com:10020136135
m.aliexpress.com:1148794713

netizeni commented 3 months ago

Same thing on my side. After cleaning stats, waited for two days and numbers are increasing again, so no change.

andylau004 commented 2 months ago

i use unbound and dnscrypt proxy upstream to quad9 resolver. but maybe you can test following changes:我在 Quad9 解析器的上游使用 Unbound 和 DNSCRYPT 代理。但也许您可以测试以下更改：

ratelimit: 0
refuse_any: false
upstream_mode: parallel
fastest_timeout: 1s
enable_dnssec: true
max_goroutines: 500
handle_ddr: true
upstream_timeout: 2s

Optional:
bootstrap_prefer_ipv6: true

I don't know ControlD and its service - but maybe you could test it without it. Your ControlD settings use OISD and Hagezi's blacklist - do you also have these lists in AGH? The third DNS service is dns0 – the nextdns eu brother, also with filter lists...我不知道 ControlD 及其服务 - 但也许您可以在没有它的情况下测试它。您的 ControlD 设置使用 OISD 和 Hagezi 的黑名单 - 您在 AGH 中也有这些名单吗？第三个 DNS 服务是 dns0 – nextdns eu 兄弟，也带有过滤器列表......

It's all doubled twice. - Its not bad, but for testing we should use only one DNS Service to exclude the problem.... Which filter lists do you use in AGH?都翻了两倍。- 这还不错，但是对于测试，我们应该只使用一个DNS服务来排除问题。您在 AGH 中使用哪些筛选器列表？

My results for the DNS Service you use:我对您使用的 DNS 服务的结果：

root@HomeNetDNS:~# mtr -r -w -c4 193.110.81.0
Start: 2024-03-16T08:53:35+0100
HOST: HomeNetDNS                                Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- Fritzbox                                   0.0%     4    0.7   0.8   0.6   0.9   0.1
  2.|-- 100.124.1.76                               0.0%     4    8.8   8.7   6.4  12.0   2.4
  3.|-- 100.127.1.133                              0.0%     4    6.8   6.7   5.6   8.5   1.3
  4.|-- 100.127.1.132                              0.0%     4    6.2   6.1   4.4   7.7   1.3
  5.|-- 185.22.46.129                              0.0%     4    5.9  20.9   5.9  37.3  16.9
  6.|-- ae3-1337.bbr02.anx63.ams.nl.anexia-it.net  0.0%     4   18.4  17.4  15.8  19.4   1.8
  7.|-- ae1-10.bbr01.anx63.ams.nl.anexia-it.net    0.0%     4   16.3  16.1  15.0  17.7   1.2
  8.|-- dns0.eu                                    0.0%     4   14.4  15.6  14.4  17.3   1.3
root@HomeNetDNS:~# mtr -r -w -c4 76.76.2.41
Start: 2024-03-16T08:55:00+0100
HOST: HomeNetDNS    Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- Fritzbox       0.0%     4    0.7   0.8   0.7   0.9   0.1
  2.|-- 100.124.1.76   0.0%     4    6.8   7.2   5.3   8.5   1.5
  3.|-- 100.127.1.132  0.0%     4    6.4   7.0   5.5   8.4   1.3
  4.|-- 185.22.46.129  0.0%     4    8.3   8.1   6.4   9.2   1.2
  5.|-- ???           100.0     4    0.0   0.0   0.0   0.0   0.0
  6.|-- ???           100.0     4    0.0   0.0   0.0   0.0   0.0
  7.|-- 76.76.2.41     0.0%     4   12.1  12.5  11.1  13.6   1.1
root@HomeNetDNS:~# mtr -r -w -c4 76.76.2.32
Start: 2024-03-16T08:55:18+0100
HOST: HomeNetDNS    Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- Fritzbox       0.0%     4    0.9   0.9   0.8   1.2   0.2
  2.|-- 100.124.1.76   0.0%     4    7.6   8.5   7.0  10.3   1.5
  3.|-- 100.127.1.131  0.0%     4    5.5   6.7   4.8   9.4   2.0
  4.|-- 100.127.1.132  0.0%     4    4.6   6.8   4.6   9.2   2.0
  5.|-- 185.22.46.145  0.0%     4    8.2   7.9   6.3   9.7   1.4
  6.|-- ???           100.0     4    0.0   0.0   0.0   0.0   0.0
  7.|-- ???           100.0     4    0.0   0.0   0.0   0.0   0.0
  8.|-- 76.76.2.32     0.0%     4   11.6  10.9   9.1  12.7   1.6
root@HomeNetDNS:~# mtr -r -w -c4 9.9.9.9
Start: 2024-03-16T08:56:15+0100
HOST: HomeNetDNS                   Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- Fritzbox                      0.0%     4    1.1   0.9   0.6   1.1   0.3
  2.|-- 100.124.1.76                  0.0%     4    5.9   7.1   5.9   8.5   1.1
  3.|-- 100.127.1.133                 0.0%     4    5.4   5.8   4.3   7.1   1.2
  4.|-- 100.127.1.132                 0.0%     4    6.7   6.7   5.8   7.8   0.9
  5.|-- 185.22.46.145                 0.0%     4    8.4   7.2   5.7   8.4   1.1
  6.|-- as42.dusseldorf.megaport.com  0.0%     4    9.1  10.4   9.1  12.0   1.2
  7.|-- dns9.quad9.net                0.0%     4    9.5  11.0   9.5  12.4   1.2

i use unbound and dnscrypt proxy upstream to quad9 resolver. but maybe you can test following changes:我在 Quad9 解析器的上游使用 Unbound 和 DNSCRYPT 代理。但也许您可以测试以下更改：我在 Quad9 解析器的上游使用 Unbound 和 DNSCRYPT 代理。但也许您可以测试以下更改：我在 Quad9 解析器的上游使用 Unbound 和 DNSCRYPT 代理。但也许您可以测试以下更改：

ratelimit: 0 refuse_any: false upstream_mode: parallel fastest_timeout: 1s enable_dnssec: true max_goroutines: 500 handle_ddr: true upstream_timeout: 2s

Optional: bootstrap_prefer_ipv6: true I don't know ControlD and its service - but maybe you could test it without it. Your ControlD settings use OISD and Hagezi's blacklist - do you also have these lists in AGH? The third DNS service is dns0 – the nextdns eu brother, also with filter lists...我不知道 ControlD 及其服务 - 但也许您可以在没有它的情况下测试它。您的 ControlD 设置使用 OISD 和 Hagezi 的黑名单 - 您在 AGH 中也有这些名单吗？第三个 DNS 服务是 dns0 – nextdns eu 兄弟，也带有过滤器列表...... 我不知道 ControlD 及其服务 - 但也许您可以在没有它的情况下测试它。您的 ControlD 设置使用 OISD 和 Hagezi 的黑名单 - 您在 AGH 中也有这些名单吗？第三个 DNS 服务是 dns0 – nextdns eu 兄弟，也带有过滤器列表......我不知道 ControlD 及其服务 - 但也许您可以在没有它的情况下测试它。您的 ControlD 设置使用 OISD 和 Hagezi 的黑名单 - 您在 AGH 中也有这些名单吗？第三个 DNS 服务是 dns0 – nextdns eu 兄弟，也带有过滤器列表......

It's all doubled twice. - Its not bad, but for testing we should use only one DNS Service to exclude the problem.... Which filter lists do you use in AGH?都翻了两倍。- 这还不错，但是对于测试，我们应该只使用一个DNS服务来排除问题。您在 AGH 中使用哪些筛选器列表？都翻了两倍。- 这还不错，但是对于测试，我们应该只使用一个DNS服务来排除问题。您在 AGH 中使用哪些筛选器列表？都翻了两倍。- 这还不错，但是对于测试，我们应该只使用一个DNS服务来排除问题。您在 AGH 中使用哪些筛选器列表？

My results for the DNS Service you use:我对您使用的 DNS 服务的结果： My results for the DNS Service you use：我对您使用的 DNS 服务的结果：

root@HomeNetDNS:~# mtr -r -w -c4 193.110.81.0 Start: 2024-03-16T08:53:35+0100 HOST: HomeNetDNS Loss% Snt Last Avg Best Wrst StDev 1.|-- Fritzbox 0.0% 4 0.7 0.8 0.6 0.9 0.1 2.|-- 100.124.1.76 0.0% 4 8.8 8.7 6.4 12.0 2.4 3.|-- 100.127.1.133 0.0% 4 6.8 6.7 5.6 8.5 1.3 4.|-- 100.127.1.132 0.0% 4 6.2 6.1 4.4 7.7 1.3 5.|-- 185.22.46.129 0.0% 4 5.9 20.9 5.9 37.3 16.9 6.|-- ae3-1337.bbr02.anx63.ams.nl.anexia-it.net 0.0% 4 18.4 17.4 15.8 19.4 1.8 7.|-- ae1-10.bbr01.anx63.ams.nl.anexia-it.net 0.0% 4 16.3 16.1 15.0 17.7 1.2 8.|-- dns0.eu 0.0% 4 14.4 15.6 14.4 17.3 1.3 root@HomeNetDNS:~# mtr -r -w -c4 76.76.2.41 Start: 2024-03-16T08:55:00+0100 HOST: HomeNetDNS Loss% Snt Last Avg Best Wrst StDev 1.|-- Fritzbox 0.0% 4 0.7 0.8 0.7 0.9 0.1 2.|-- 100.124.1.76 0.0% 4 6.8 7.2 5.3 8.5 1.5 3.|-- 100.127.1.132 0.0% 4 6.4 7.0 5.5 8.4 1.3 4.|-- 185.22.46.129 0.0% 4 8.3 8.1 6.4 9.2 1.2 5.|-- ??? 100.0 4 0.0 0.0 0.0 0.0 0.0 6.|-- ??? 100.0 4 0.0 0.0 0.0 0.0 0.0 7.|-- 76.76.2.41 0.0% 4 12.1 12.5 11.1 13.6 1.1 root@HomeNetDNS:~# mtr -r -w -c4 76.76.2.32 Start: 2024-03-16T08:55:18+0100 HOST: HomeNetDNS Loss% Snt Last Avg Best Wrst StDev 1.|-- Fritzbox 0.0% 4 0.9 0.9 0.8 1.2 0.2 2.|-- 100.124.1.76 0.0% 4 7.6 8.5 7.0 10.3 1.5 3.|-- 100.127.1.131 0.0% 4 5.5 6.7 4.8 9.4 2.0 4.|-- 100.127.1.132 0.0% 4 4.6 6.8 4.6 9.2 2.0 5.|-- 185.22.46.145 0.0% 4 8.2 7.9 6.3 9.7 1.4 6.|-- ??? 100.0 4 0.0 0.0 0.0 0.0 0.0 7.|-- ??? 100.0 4 0.0 0.0 0.0 0.0 0.0 8.|-- 76.76.2.32 0.0% 4 11.6 10.9 9.1 12.7 1.6 root@HomeNetDNS:~# mtr -r -w -c4 9.9.9.9 Start: 2024-03-16T08:56:15+0100 HOST: HomeNetDNS Loss% Snt Last Avg Best Wrst StDev 1.|-- Fritzbox 0.0% 4 1.1 0.9 0.6 1.1 0.3 2.|-- 100.124.1.76 0.0% 4 5.9 7.1 5.9 8.5 1.1 3.|-- 100.127.1.133 0.0% 4 5.4 5.8 4.3 7.1 1.2 4.|-- 100.127.1.132 0.0% 4 6.7 6.7 5.8 7.8 0.9 5.|-- 185.22.46.145 0.0% 4 8.4 7.2 5.7 8.4 1.1 6.|-- as42.dusseldorf.megaport.com 0.0% 4 9.1 10.4 9.1 12.0 1.2 7.|-- dns9.quad9.net 0.0% 4 9.5 11.0 9.5 12.4 1.2

i use unbound and dnscrypt proxy upstream to quad9 resolver. but maybe you can test following changes:
ratelimit: 0
refuse_any: false
upstream_mode: parallel
fastest_timeout: 1s
enable_dnssec: true
max_goroutines: 500
handle_ddr: true
upstream_timeout: 2s

Optional:
bootstrap_prefer_ipv6: true

thks for you reply. 谢谢你回复。

Especially below config items, and i new add multi upstream dns server address, The upstream response more faster

ratelimit: 0 refuse_any: false upstream_mode: parallel fastest_timeout: 1s enable_dnssec: true max_goroutines: 500 handle_ddr: true upstream_timeout: 2s

@whyisthisbroken

andylau004 commented 2 months ago

login.aliexpress.com:10027762671 a1931.dscgi3.akamai.net:10024301298 cdn.smoot.g.aaplimg.com:10024288682 a1931.dscgi3.akamai.net:10021771928

measuring unit is Nanosecond?? just like that: login.aliexpress.com:10027762671 ns??

dansseg commented 2 months ago

When did this issue appear for you guys? For me this started with the version released in February. Did someone do a test with an older version already? (Migrating back is not that easy, as the schema version changed -> manual steps needed) Using Pi-hole (with same DNS server) I do not have issues.

YMan84 commented 2 months ago

Restoring a backup of my Adguard Home Proxmox VM to version 0.107.43 from 0.107.48 has solved the issue for me.

Upstream Latency with Verison 0.107.48 https://dns10.quad9.net:443/dns-query 1778 ms

Upstream Latency with Version 0.107.43 https://dns10.quad9.net:443/dns-query 82 ms

Sorry i can't be of more help than that.

dMopp commented 2 months ago

Thanks already. All I can find are many go version bumps and this one: https://github.com/AdguardTeam/AdGuardHome/issues/6480

Everything else from the changelog seems to be unrelated for my eyes

gigi2006 commented 2 months ago

I have exactly the same problem and also think that it is due to the version. Unfortunately I don't have a backup.

I hope the problem will be fixed soon.

Restoring a backup of my Adguard Home Proxmox VM to version 0.107.43 from 0.107.48 has solved the issue for me.

Upstream Latency with Verison 0.107.48 https://dns10.quad9.net:443/dns-query 1778 ms

Upstream Latency with Version 0.107.43 https://dns10.quad9.net:443/dns-query 82 ms

Sorry i can't be of more help than that.

ingoratsdorf commented 2 months ago

Hi guys, same issue here. Running a standard docker install in host networking mode, version v0.107.48. It's running okay for a few hours, came back this morning to find crazy numbers in the average upstream response time. Resetting statistics makes it look okay again, but surely there are some issues.

cat querylog.json | jq -r '(.QH + ":" + (.Elapsed | tostring))' | sort -t: -nrk2 | head -20 audio-fa.scdn.co:350989851159 proxy.safebrowsing.apple:344889509624 gateway.instagram.com:343834699734 safebrowsing-proxy.g.aaplimg.com:341729084693 proxy.safebrowsing.apple:338749865929 ap.spotify.com:338208111756 api.fe.amazonalexa.com:338044546340 www.gstatic.com:337437600395 a1rabvci4qcikc.us-east-1.prod.service.minerva.devices.a2z.com:337381242824 safebrowsing-proxy.g.aaplimg.com:337225867429 safebrowsing-proxy.g.aaplimg.com:334874724649 proxy.safebrowsing.apple:334557784285 safebrowsing-proxy.g.aaplimg.com:332885854272 ap-guc3.spotify.com:332840456493 api.fe.amazonalexa.com:332045441820 a1rabvci4qcikc.us-east-1.prod.service.minerva.devices.a2z.com:331427295611 api.fe.amazonalexa.com:326556549968 m2.tuyaus.com:326397286578 ap-guc3.spotify.com:323510000868 ap.spotify.com:320207516550

ingoratsdorf commented 2 months ago

Based on that find, I have entered my routers IP into "Private reverse DNS servers" and upgraded back to "v0.107.48" and the issue is gone. Running successfully for more than 24 hrs and all upstream DNS server response times are going down and not up like they did before.

Average upstream response time for the last 24 hours 9.9.9.9:53 49 ms 1.1.1.1:53 25 ms 122.56.237.1:53 14 ms 192.168.1.254:53 13 ms 192.168.1.1:5053 1 ms

dMopp commented 1 month ago

Why closed ?

netizeni commented 1 month ago

@overwatch3560 I have the same question as @dMopp, why this issue is closed and completed, when the issue is still present? Provided workarounds didn't fix the issue, and they are not solutions anyway.

overwatch3560 commented 1 month ago

Restoring a backup of my Adguard Home Proxmox VM to version 0.107.43 from 0.107.48 has solved the issue for me.

Upstream Latency with Verison 0.107.48

https://dns10.quad9.net:443/dns-query 1778 ms

Upstream Latency with Version 0.107.43

https://dns10.quad9.net:443/dns-query 82 ms

Sorry i can't be of more help than that.

I thought it was solved using this solution

ingoratsdorf commented 1 month ago

Restoring a backup of my Adguard Home Proxmox VM to version 0.107.43 from 0.107.48 has solved the issue for me. Upstream Latency with Verison 0.107.48 https://dns10.quad9.net:443/dns-query 1778 ms Upstream Latency with Version 0.107.43 https://dns10.quad9.net:443/dns-query 82 ms Sorry i can't be of more help than that.

I thought it was solved using this solution

Not a solution to downgrade. It is just an indicator that the current version has an issue and it's not a user's system issue. I downgraded as well as others and that solved it for me as well. But then I read all other open issues Re speed and tried some of the solutions and the one I noted above solved the issue for me on the current version.

netizeni commented 1 month ago

Restoring a backup of my Adguard Home Proxmox VM to version 0.107.43 from 0.107.48 has solved the issue for me. Upstream Latency with Verison 0.107.48 https://dns10.quad9.net:443/dns-query 1778 ms Upstream Latency with Version 0.107.43 https://dns10.quad9.net:443/dns-query 82 ms Sorry i can't be of more help than that.

I thought it was solved using this solution

As @ingoratsdorf mentioned, downgrading the version is definitely not a solution, so please re-open this issue, as it exists in the current latest version.

AdguardTeam / AdGuardHome