AdguardTeam / AdGuardHome

Network-wide ads & trackers blocking DNS server
https://adguard.com/adguard-home/overview.html
GNU General Public License v3.0
25.57k stars 1.84k forks source link

Update v0.107.30 causes connection and speed drops. #5874

Open Ice2192 opened 1 year ago

Ice2192 commented 1 year ago

Prerequisites

Operating system type

Windows

CPU architecture

64-bit ARM

Installation

GitHub releases or script from README

Setup

On a router, DHCP is handled by the router

AdGuard Home version

v0.107.30

Description

What did you do?

Go on the internet.

Expected result

View pages.

Actual result

Error: "This Site Can’t Be Reached"

Screenshots (if applicable)

Additional information

I just updated ASAP when I got the notification today and then the internet just drops. I thought it was the router but I changed the DNS on my PC and it was fine. I recently upgraded from a miro center sd card to a sandisk a few months ago as the microcenter one hangs when updating the blocklists for some reason. Had no problems since the san disk upgrade. Now with this update, the DNS just gets lost when I point anything to my pi running adguard home.

Ice2192 commented 1 year ago

Thanks! It did stop AGH however when I tried to copy the yaml file with the verbose edit via filezilla, I gave me a permissions error. I don't want to waste even more of your time, as you've been very patient with me thus far, and ask how to remove it so I pasted it the old fashioned way of turning off the pi and taking out the sd card and paste it via my nas as it's the only other way i could view the files.

Do I just connect devices to the AGH and visit websites for the thing to input information into the log and then take out the log and email you guys?

EugeneOne1 commented 1 year ago

@Ice2192, that's right. It would be great, if the actual issue case appeared and have been captured. Thanks for your engagement.

Ice2192 commented 1 year ago

@EugeneOne1 I sent it.

EugeneOne1 commented 1 year ago

@Ice2192, hello again. We've inspected your log and unfortunately found nothing suspicious. Could you please provide some details:

It also may be related to the filtering. Could you please try to disable the protection for some time and check if the problem persists? And also, could you please filter the query log by blocked status and check if something legal have been blocked there?

Ice2192 commented 1 year ago

@EugeneOne1 Any app / website doesn't work. If I go to any website, it just either hangs in the loading screen or I get an error as mentioned in the first post. It was working fine until .30 update came along. Haven't changed any blocklist since then. I usually get a ping of 2 but the update causes the ping to be ≥ 5. I used to use a microcenter sd card that they gave away for free but those causes the pi to hang when it needed to update the blocklist which I would have to force restart the pi everytime. Then I got a san disk one and since then haven't had problems until now. Hard resetting to clear cache for both my routers still didn't change a thing. Right now every time I point my router to the pi it works for like an hour or 2 and then just slows to a halt.

Ice2192 commented 1 year ago

@EugeneOne1 Is there a way to downgrade to 29. Anything after that version completely made everything slow. I even did a clean wipe of the sd card and reinstalled Pi OS and AGH .

boarder010 commented 1 year ago

FYI: having the same issue - all was well with v29 and ever since then I've had to turn off AGH because it noticeably slows down traffic. on v32 all seems well for the first few minutes and then eventually crawls. I uninstalled/re-installed through homebridge with same result.

Screenshot 2023-06-16 at 7 24 11 AM
danieletorelli commented 1 year ago

FYI: having the same issue - all was well with v29 and ever since then I've had to turn off AGH because it noticeably slows down traffic. on v32 all seems well for the first few minutes and then eventually crawls. I uninstalled/re-installed through homebridge with same result.

Screenshot 2023-06-16 at 7 24 11 AM

@boarder010 Did you disable only the AGH protection from the button in the main page or the entire service switching to different DNS?

Mur1thi commented 1 year ago

Experiencing the same with the latest version too. AdGuard home works well in the first few minutes then it breaks and I'm no longer connected to the internet.

boarder010 commented 1 year ago

@danieletorelli > I had to switch back to the ISP provided DNS at the router level in order to get things back to normal speed

Xerdunits commented 1 year ago

@boarder010 @Mur1thi Why does the localhost show up in the log though? Does AHG only provide filtering and the actual DNS server is your router?What DNS server are you guys using, how does your setup look? Do you use multiple DNS server, or just one?

Mind sharing your DNS settings?

boarder010 commented 1 year ago

@Xerdunits - router points to AGH IP and only using 1 dns server (dns.adguard-dns.com/dns-query), also gave quad9 a try with same result.

V29 and earlier worked perfectly with no issues until v30 upgrade, no changes to router or agh settings. No clue home to downgrade in homebridge otherwise I’d give it a shot hah

lbm commented 1 year ago

I'm also experiencing the same slowdowns on the latest version (v0.107.29 was the last working version). This is on the ARM image running on a Kubernetes cluster.

danieletorelli commented 1 year ago

The issue is still present in the last release.

It seems like suddenly AGH takes long time to respond and this doesn't only happen in DNS queries but also on the API, since I see the timeouts again on HomeAssistant plugin as well. So it's a generalized performance degradation of the service.

Rolling back again to 29 at the moment.

Xerdunits commented 1 year ago

@boarder010 What does the 'average processing time' say? It should be very high in that case I imagine.

boarder010 commented 1 year ago

@Xerdunits - my avg can’t be used since I just reinstalled and as soon as i turned it on I knew it wasn’t fixed, so I stopped using it

Xerdunits commented 1 year ago

@boarder010 Did you start with a fresh config, or your old one? What filters do you use? Are you using the same DNS server with AGH disabled?

chaoscalm commented 1 year ago

https://dns10.quad9.net/dns-query

For me quad9 was the source of the problem, I moved to Google during an hang of the dns resolver and it returned operational, in my opinion the choosing resolver mechanism isn't capable of choose correctly where to resolve.

Also, I noticed that dns resolver was the only one being the default.

https://github.com/KelvinTegelaar/CIPP/issues/1352

Ice2192 commented 1 year ago

@chaoscalm I'll try that. I just turned mines on last night to see if there is a difference. There are patches in time where the response time is 35k ms but the majority of the time its decent. I'll see if that works.

chaoscalm commented 1 year ago

@chaoscalm I'll try that. I just turned mines on last night to see if there is a difference. There are patches in time where the response time is 35k ms but the majority of the time its decent. I'll see if that works.

I sometime get over 100k ms replies, I don't think this is a fix, but if quad9 rate-limit, it can make it slightly better, I got some servfail even with google getting 999 ms, so I don't know what's going on.

Ice2192 commented 1 year ago

@chaoscalm So far I'm getting sub ~30ms and every site im visiting is just snappier than usual. I followed a NetworkChuck video and he added the quad9 thing and based on your previous post I might just take your advice as a test. I'm open to anything at this point. I tried googling how to downgrade adguard home but dont see any results and if there are its people who are using a different method as I did like docker which I dont use. I tried asking in this thread but no one has responded yet. Version 29 is still the most stable build in my book.

ppfeufer commented 1 year ago

Ever tried using unbound as your DNS resolver? » https://dev.to/cipherops/how-to-use-unbound-with-adguard-home-1o5n

I'm doing so running AGH and unbound on a Raspi 4 and have no issues at all and it's quite fast … image

Although I'm using a slightly modified unbound config, feel free to test it:

/etc/unbound/unbound.conf.d/config.conf

server:
  interface: 127.0.0.1
  interface: ::1
  port: 5335

  do-ip6: yes
  prefer-ip6: yes
  do-ip4: yes
  do-udp: yes
  do-tcp: yes

  # Set number of threads to use
  num-threads: 4

  # Log settings
  log-queries: no
  log-replies: no
  log-servfail: no
  log-local-actions: no
  logfile: /dev/null

  # Hide DNS Server info
  hide-identity: yes
  hide-version: yes

  # Limit DNS Fraud and use DNSSEC
  harden-algo-downgrade: yes
  harden-below-nxdomain: yes
  harden-large-queries: yes
  harden-glue: yes
  harden-dnssec-stripped: yes
  harden-referral-path: yes
  harden-short-bufsize: yes
  use-caps-for-id: yes
  qname-minimisation: yes
  deny-any: yes
  aggressive-nsec: yes
  rrset-roundrobin: yes
  minimal-responses: yes

  # If DNSSEC isnt working uncomment the following line
#  auto-trust-anchor-file: "/var/lib/unbound/root.key"

  # Minimum lifetime of cache entries in seconds
  cache-min-ttl: 1800

  # Configure TTL of Cache
  cache-max-ttl: 3600

  # Optimizations
  msg-cache-slabs: 8
  rrset-cache-slabs: 8
  infra-cache-slabs: 8
  key-cache-slabs: 8
  outgoing-range: 8192
  num-queries-per-thread: 4096
  serve-expired: yes
  serve-expired-ttl: 3600
  edns-buffer-size: 1232
  prefetch: yes
  prefetch-key: yes
  target-fetch-policy: "3 2 1 1 1"
  unwanted-reply-threshold: 10000000
  so-sndbuf: 4m
  so-rcvbuf: 4m
  so-reuseport: yes
  # Set cache size
  rrset-cache-size: 2048m
  msg-cache-size: 1024m
  neg-cache-size: 4m

  # increase buffer size so that no messages are lost in traffic spikes
  so-rcvbuf: 1m

  # Private Networks
  private-address: 192.168.0.0/16
  private-address: 169.254.0.0/16
  private-address: 172.16.0.0/12
  private-address: 10.0.0.0/8
  private-address: fd00::/8
  private-address: fe80::/10
chaoscalm commented 1 year ago

Seems that so far enabling also the Optimistic caching, reduce the timeout.

Ice2192 commented 1 year ago

@ppfeufer Holy cow dude! I took your unbound advice and this is what I got: Screenshot 2023-06-18 143619 There are some requests that are 10K ms response which is throwing off the avg but everything works as if I had the adguard off.

ppfeufer commented 1 year ago

Clear AGH's cache and make sure unbound is the only upstream DNS you have.

Ice2192 commented 1 year ago

Clear AGH's cache and make sure unbound is the only upstream DNS you have.

I did that but I went to: https://d3ward.github.io/toolz/adblock.html and got a 5% rating.

https://adblock-tester.com/ got a 49/100 rating.

I went to the worst ad offenders like fandom and IGN anon my phone d the ads are still appearing there. Is there any more additional settings I need to change.

ppfeufer commented 1 year ago

Erm, that's a matter of your block lists. You can, if you like, use mine, but that's a pretty restrictive one. (https://github.com/ppfeufer/adguard-filter-list)

image Doesn't even tell me which 4 are not blocked, so I take that as a somewhat good sign.

image

ppfeufer commented 1 year ago

If you opt to use my block list, make sure it's the only active one, because it combines a metric ton of other lists into one and the other lists you have active are probably in there as well.

Ice2192 commented 1 year ago

But I had 90%-95% before the change. I'm not sure if Unbound is actually working in terms of helping blocking ads. It's fast as when I turned off the ad blocker but maybe because it isn't blocking ads at all. Is there a line of code to revert back to 29?

ppfeufer commented 1 year ago

You're misunderstanding what unbound is. unbound is just a DNS resolver, which AGH is using if AGH is configured to do so. unbound does not block anything, that's AGH's job. AGH does so via block lists you can configure. These block lists are subject to change, meaning, whoever maintains them also changes them — adding or removing items. AGH will update these lists periodically, which results in AGH having fewer items to block today than it had yesterday, tomorrow it might be again a different story.

All unbound does is resolve names that AGH throws at it and return their IP addresses to AGH.

Ice2192 commented 1 year ago

Thanks for the clarification. The version 29 is still on the GitHub I see. How do I tell my raspberry pi to download that rather than the latest update?

ppfeufer commented 1 year ago

That I don't know.

But, if your speed issue is resolved by using unbound, I highly recommend checking your block lists for updates. (Or try mine and whitelist what you need and is blocked by it).

Another point of error might be that unbound is listening on port 53, which it should not. Check that as well just for good measures, because if that is the case, AGH does not block anything because it's out of the loop.

ppfeufer commented 1 year ago

The /etc/resolv.conf file on your AGH Raspi should look like this:

nameserver fd00::2
nameserver 10.10.10.2

Just replace the IPs with the ones your AGH Raspi has.

ppfeufer commented 1 year ago

Also, check the DNS configuration of your devices and restart them (with the AGH Raspi being the first). They still might work with a cached DNS configuration you eventually have entered for testing purposes.

Xerdunits commented 1 year ago

I'm honestly not sure if this isn't just a coincidence with the update and not an issue with AGH in itself. And I'm also not sure why you would setup AGH and not using any of the features it offers, but that just me ;)

Mur1thi commented 1 year ago

@boarder010 @Mur1thi Why does the localhost show up in the log though? Does AHG only provide filtering and the actual DNS server is your router?What DNS server are you guys using, how does your setup look? Do you use multiple DNS server, or just one?

Mind sharing your DNS settings?

My router points to one ADGH IP address. I'm running ADGH on a raspberry pi. After a few min of having ADGH as the DNS server, I'm completely cut off from the internet. I only have 2 cloudflare IPs as upstream. After that, my query log looks like this. see image

image
boarder010 commented 1 year ago

@Ice2192 @Mur1thi - I think I may have figured out the issue...something you may want to check. I had a clean reinstall, everything processes quickly but it would just hang on actually loading the page. Looking at all the settings again, I noticed "use private reverse dns resolver" was checked even though I didn't have anything there entered. Unchecked, applied, and everything has been running as normal for the past 30 minutes at least

Screenshot 2023-06-20 at 8 07 47 AM
Ice2192 commented 1 year ago

@boarder010 Keep us updated. I have clean installed and followed ppfeufer's instructions (3 times) to the T and I'm just not getting the same results in terms of adblocking. It's either speed but no blocking (which is the same as not having AGH on) or speed of 56k modem with adblocking. I somehow can't get both. If I do get both, it'll last for about an hour or so or I need to constantly use the internet because any downtime of not using it will go back to slow speeds. It's like every time I have to use the internet again I have to make AGH stretch for a few minutes before it can exercise.

boarder010 commented 1 year ago

@Ice2192 - I'm an hour in now and all good still just like v29. My Adblock score was 99% and my ADGH avg processing time is around 23ms. I just have router dns pointing to ADGH and ADGH is only using one cloudfare upstream.

chaoscalm commented 1 year ago

Hello boarder010, I am testing what you have described, and since 1 hour I am not having problems anymore.

Thank you for sharing your fix.

Best Regards, chaoscalm

Ice2192 commented 1 year ago

Check Up Average processing time: 85 ms Scores: https://d3ward.github.io/toolz/adblock.html - 93% https://adblock-tester.com/ - 80/100

Ice2192 commented 1 year ago

If I toggle the counting of "Banner advertising" to off in the adblock tester website I have a full 100/100. Anyone knows how to block those?

Mur1thi commented 1 year ago

@Ice2192 @Mur1thi - I think I may have figured out the issue...something you may want to check. I had a clean reinstall, everything processes quickly but it would just hang on actually loading the page. Looking at all the settings again, I noticed "use private reverse dns resolver" was checked even though I didn't have anything there entered. Unchecked, applied, and everything has been running as normal for the past 30 minutes at least

Screenshot 2023-06-20 at 8 07 47 AM

This fix is working well for me, thanks a lot

webocalypse commented 1 year ago

Hi everyone,

i ran into the same issue as some of you did. For context, i run adguardhome via docker/docker-compose and the only upstream is unbound. Unbound has a hostname "unbound" und my configured upstream is udp://unbound. With version 0.107.30 dns queries to unbound seemed to fail. Also version .31 and .32 did not fix it for me. A friend of mine had the issue as well and he found that when validating upstreams, udp://unbound could not be resolved. So i checked the changelog and found that since version .30 upstream hostnames are being resolved via bootstrap dns servers. In consequence the internal docker dns was not used resulting in not resolving unbound correctly.

So is added

udp://127.0.0.11:53

as bootstrap dns and everything is currently working fine. DNS is being resolved via unbound again. It's running for like 10 minutes.

I hope this possible solution might help some of you.

ainar-g commented 1 year ago

@EugeneOne1, I feel like this could be because of query recursion not being detected. Please investigate if any changes could have broken that.

Ice2192 commented 1 year ago

@boarder010 Yup that did it. A day later and I'm at 30ms WITH actual ad blocking. Thanks!

ainar-g commented 1 year ago

@Ice2192, we'd like to keep the issue open until we find the actual reason behind the bug, if you don't mind. Having a workaround is good, but the bug is still there.

Ice2192 commented 1 year ago

@ainar-g No worries!

Mur1thi commented 1 year ago

My AGH has stopped working again. Unable to resolve DNS

Freebase394 commented 1 year ago

Hello everyone, My dear mate @Ice2192, I can see that you are having a nice results with bolocking the most of your adds.. Could you share your special configs | lists for that please ?

It would be great to share such knowledge as yours! On https://d3ward.github.io/toolz/adblock.html sometimes i get image And on https://adblock-tester.com/ Others times i get 50% and others i get 77% when my AdGuardHome version v0.107.33 is running fine due to new issue #5964 image