AdguardTeam / AdguardForiOS

The most advanced ad blocker for iOS
https://adguard.com/
GNU General Public License v3.0
1.46k stars 203 forks source link

DNS filtering failure on iOS 14.5 / 14.6 #1755

Closed dnovitskyi closed 3 years ago

dnovitskyi commented 3 years ago

Issue Details

Expected Behavior

AdGuard for iOS should be able to work on newer iOS builds

Actual Behavior

AdGuard stops working without any app config changes (apart from iOS version change)

Screenshots

N/A

Additional Information

Debug Logs: See helpdesk ticket Z504270.

cooperbang commented 3 years ago

Updating filter lists doesn´t work too with iOS 14.5. Process hangs in a loop...

Eugene-Savenko commented 3 years ago

Z504270:

It appears adguard split-tunnel mode is the culprit in 14.5. I have turned on full-tunnel mode and it is working again.

Aydinv13 commented 3 years ago

@cooperbang can you please check it on your side?

cooperbang commented 3 years ago

I only use Safari protection... filtering doesn´t work and updating filter lists doesn´t work too.

Aydinv13 commented 3 years ago

@cooperbang specify please: 1) Which version of AdGuard do you use? 2) If you talk about Safari Protection what filters you have enabled? 3) Make sure there aren't any rules (in user rules and allowlist) that prevent filtering 4) What if you add ||example.org^ in user rules and visit example.org? In case it's not blocked send your log to us please apple@adguard.com

cooperbang commented 3 years ago
  1. version 4.1.1(668)
  2. only default installation filters Filters subscriptions: ID=2 Name="AdGuard Base filter" Version=2.1.74.0 Enabled=1 ID=11 Name="AdGuard Mobile Ads filter" Version=2.0.19.24 Enabled=1 ID=3 Name="AdGuard Tracking Protection filter" Version=2.0.23.92 Enabled=1 ID=6 Name="AdGuard German filter" Version=2.0.18.50 Enabled=1
  3. no user rules and allowlist
  4. if I add ||exampe.org^ the site is blocked by Safari

I think the problem is that sometimes the filter updating process takes a lot of time (several minutes) - sometimes it´s normal and everything works (including filtering).

Aydinv13 commented 3 years ago

@cooperbang Could you turn on Debug logs then click on update again and send your logs to us? Also tell the definite time you were updating filters

nekosama310 commented 3 years ago

Before that I used ios 14.5 and used the Youtube app it was blocked for ads, but after upgrading ios 14.6 with the same filter, my Youtube ad appeared again

cooperbang commented 3 years ago

@cooperbang Could you turn on Debug logs then click on update again and send your logs to us? Also tell the definite time you were updating filters

I´ve sent the logfiles to apple@adguard.com. Updating filters at about 17:26.

Filtering basically works but suddenly stops after some time. Updating filter lists takes about 1min sometimes - even if there are no updates available.

maciboy commented 3 years ago

For me updating usually takes 4-5 minutes since iOS 14.5 :( I mailed the logs just now…

sdugoten commented 3 years ago

It seems like when you set Tunnel mode to split tunnel, IOS 14.5 wouldn't' work correctly. From time to time, ads are not blocked.

maciboy commented 3 years ago

I dont have the Premium Version! Im just using Safari filter lists, no VPN tunneling.

tate1435 commented 3 years ago

I’ve noticed ads slipping thru in apps since 14.5 but the one issue I just noticed is that when in full tunnel mode I can’t stream my Logitech Circle View Doorbell. It uses iCloud HomeKit secure video. When I switch back to Split Tunnel the video stream is fine.

And in split tunnel ads slip thru. Full tunnel seems to block ads but like I said I can’t stream the doorbell in HomeKit

worldsdream commented 3 years ago

For me updating usually takes 4-5 minutes since iOS 14.5 :( I mailed the logs just now…

Same here. It takes more time when you update filters after upgrading to iOS 14.5. It happened to all my devices after upgrading to 14.5.

I hope the team can find the issue and give an update to Adguard iOS app.

Trippersgarage commented 3 years ago

Same issue here. Since updating to 14.5, adguard does not update the filters; I get stuck at “rules are converting”.

iDjay commented 3 years ago

Can confirm that AdGuard doesn‘t work well with iOS 14.5.

tomtom53 commented 3 years ago

Can confirm that AdGuard doesn‘t work well with iOS 14.5.

Yes with 14.5 AdGuard don’t block correctly ads in safari or application (games…). I hope for a fix soon

paulsondervan commented 3 years ago

AdGuard no longer blocks ads in Safari on my iPad Pro model A1701 from 2017 with iOS 14.5.1.

Schermafbeelding 2021-05-23 om 11 24 54

I have completely reset the iPad but the result was the same. No blocking of ads in Safari. Strange thing is that AdGuard works fine on my iPhone 11 also with iOS 14.5.1.

ameshkov commented 3 years ago

@paulsondervan

I have completely reset the iPad

What do you mean? Have you reset it to system defaults?

Here's what we know about this:

  1. Sometimes (especially, when Apple makes any changes to Content Blocking API in the update) Safari may "forget" about existing content blockers.
  2. If this happens, the way to go it:
paulsondervan commented 3 years ago

What do you mean? Have you reset it to system defaults?

Yes, that is exactly what I have done and after that I restored a backup from iCloud.

I have carried out above advice but the ads are still there.

ameshkov commented 3 years ago

@paulsondervan is it on all websites or just macrumors is affected?

Try doing the following: General -> Advanced -> Content blockers What status of content blockers do you see on that screen?

paulsondervan commented 3 years ago

It is on all websites that have ads.

Schermafbeelding 2021-05-23 om 17 10 49

Sorry, but my iOS is in Dutch.

ameshkov commented 3 years ago

Huh, so it's all good on AdGuard's side. Tbh, I don't understand what's wrong and what's worse we don't have a test iPad Pro to check it. We'll try an emulator and see how it goes.

sdugoten commented 3 years ago

@paulsondervan

I have completely reset the iPad

What do you mean? Have you reset it to system defaults?

Here's what we know about this:

  1. Sometimes (especially, when Apple makes any changes to Content Blocking API in the update) Safari may "forget" about existing content blockers.
  2. If this happens, the way to go it:
  • Go to Safari settings, disable all content blockers
  • Wait for 10 secs (waiting is important)
  • Enable them al back
  • Wait for 10-20 secs again
  • Go to AdGuard and check filters updates, wait until updates are applied

Hi Ameshkov,

Just for your information, IOS 14.5 or anything above break Adguard Pro on IOS. Even if you enable the DNS filter and block list in DNS protection, none of them works on IOS 14.5. Some ad will show in any 3rd party app from time to time.

ameshkov commented 3 years ago

Just for your information, IOS 14.5 or anything above break Adguard Pro on IOS. Even if you enable the DNS filter and block list in DNS protection, none of them works on IOS 14.5. Some ad will show in any 3rd party app from time to time.

It's not a regular issue, but we did receive a few reports about that. No clue what's triggering it so far and why only a share of users are affected.

The simple solution for DNS issues would be to switch to the Full tunnel mode in AdGuard low-level settings.

sdugoten commented 3 years ago

Just for your information, IOS 14.5 or anything above break Adguard Pro on IOS. Even if you enable the DNS filter and block list in DNS protection, none of them works on IOS 14.5. Some ad will show in any 3rd party app from time to time.

It's not a regular issue, but we did receive a few reports about that. No clue what's triggering it so far and why only a share of users are affected.

The simple solution for DNS issues would be to switch to the Full tunnel mode in AdGuard low-level settings.

You can 100% simulate the issue by switching the "Tunnel mode" to Split tunnel. The problem of using full tunnel mode is that a lot of people use NordVPN or other VPN app with Adguard pro. If you enable full tunnel mode, you can't use any other VPN app with Adguard pro.

ameshkov commented 3 years ago

You can 100% simulate the issue by switching the "Tunnel mode" to Split tunnel

Unfortunately, we 100% can not simulate or reproduce this issue whichever mode we're using. My information is based only on users' reports we're receiving.

One more thing to try - add local blocklists as per this instruction and switch AdGuard to a different "blocking mode", try "null IP" instead of the default one.

ViRb3 commented 3 years ago

iPhone 12 user here, running iOS 14.5.1. I can confirm that after updating from iOS 14.4.2 to iOS 14.5 and beyond, custom DNS and DNS filtering completely stopped working in AdGuard Pro, but Safari filtering is still intact. Here are my results from playing with the Advanced settings:

It seems like something changed in the new iOS update, as even the app "Adblock" on the App Store recently released an update specifying "iOS 14.5 fixes". I am happy to help you diagnose this issue further, please let me know how as I really rely on AdGuard. Many thanks.

EDIT: I do have multiple VPN profiles in the iOS VPN screen - AdGuard's, and then a WireGuard one. I don't know if this makes any difference, but may be worth testing.

EDIT2: With "Full tunnel" mode now all apps that use the internet started asking me for permission to connect to local network and devices.

ameshkov commented 3 years ago

@ViRb3 please do the following:

  1. Enable "debug logging" in AdGuard's advanced settings
  2. Add example.org to the blocklist
  3. Try opening it in the browser
  4. Export logs and send them to us: devteam@adguard.com, also mention this issue in the email.
ViRb3 commented 3 years ago

@ameshkov E-mail has been sent. From my brief look over the logs, it seems like the request is correctly captured and flagged for blocking, but the block does not end up working.

EDIT: I forgot to mention, I reset all settings to default before taking the logs.

ameshkov commented 3 years ago

@ViRb3 could you please try the latest beta version we've published a couple of days ago?

https://agrd.io/adguard_for_ios_testflight

ViRb3 commented 3 years ago

Hello @ameshkov. I just installed AdGuard 4.2.0(699), latest from TestFlight at the time of writing. There is no difference compared to the stable version. I have new observations, though.

Before I sent the debug logs, you told me to add "example.org" to the blocklist and see what happens. My results were that it didn't get blocked. However, upon a closer look, it appears that Safari cached something - I get redirected to "www.example.org", even if I explicitly input "http://example.org". Opening a Private tab seems to ignore this cache, because now I get the expected error: "Safari cannot open the page because the server cannot be found". Also, if I add "www.example.org" to the blocklist as well, it gets blocked even in non-Private mode. So, the blocklist definitely works.

But what doesn't work? I test against the free game Slices with the AdGuard DNS Filter. Prior to iOS 14.5, this exact filter list and game worked great. Now, with Split tunnel, 10% of the bottom of the screen is an ad window. With Full tunnel, there is no ad window. What's interesting is that even though there is an ad window in Split tunnel, there is actually no ad inside it. Clicking on the ad window attempts to open an ad in Safari, but I get the "server cannot be found" error. Note that I disabled AdGuard's Safari filtering entirely before testing this, so the blocking definitely comes from the DNS filter. Is it possible that certain parts of the app bypass the VPN in Split tunnel mode? I would encourage you to try and reproduce this setup, you should see the same behavior. I am happy to take more logs if you think that's necessary.

Many thanks for the support

EDIT: With just the DNS filter enabled, casually surfing through Safari shows no ads at all. The non-blocking issue appears to be only with some native apps.

ameshkov commented 3 years ago

Oh, well, it's great news that blocking works as intended.

Regarding this Slices app, we'll take a closer look: https://github.com/AdguardTeam/DnsLibs/issues/110

I've tried testing it myself and I don't see ads in the bottom of the screen, but I see some ad interstitials. Were they blocked before?

Aydinv13 commented 3 years ago

Didn't face an ad in the bottom of the screen in "Slices" too

ViRb3 commented 3 years ago

@ameshkov With Split tunnel, I get both ads on the bottom of the screen and ad interstitials (full screen ads after some time). With Full tunnel, I get neither. Note that it sometimes takes up to a minute for the bottom ads to appear. Screenshot attached for the "ads on bottom":

ameshkov commented 3 years ago

Okay, took us some time to properly research this topic.

It seems that there's indeed a bug, and this bug was introduced in iOS 14.5 or 14.6. It seems that some apps do not follow DNS settings of the VPN tunnel and send the second request to the system default DNS server as well. This report on Apple's dev forum seems relevant.

The conditions are unclear, it seems absolutely random. For instance, I cannot reproduce it on my own device, but it's easily reproducible on a different test phone.

We'll file a bug report to Apple and see what they think about this. This does look like an iOS bug, but it should be reported very carefully, Apple is reluctant to fixing anything VPN+DNS-related.

ViRb3 commented 3 years ago

Very glad to hear you finally managed to reproduce the issue! Please keep us updated on the query to Apple, I sincerely hope they fix the problem. I also want to take a moment and thank you and your team for the relentless support and transparency, I really admire that.

ViRb3 commented 3 years ago

@ameshk Actually, two more observations.

Have you tested if Split tunnel mode actually works with "Personal VPN" apps? I tried very hard to get it to work with my WireGuard VPN, but I could only choose one or the other, not both. I may be misunderstanding how it works, for which I apologize.

Since Split Tunnel behaves like Full Tunnel in my use case, I absolutely don't mind using the latter. However, as I mentioned before, I get constant prompts to allow each app to connect to network devices. I suppose this is because you run the VPN on an IP different than localhost, so IOS thinks it's on the network. Would it be possible to make Full Tunnel avoid this prompt? Other apps like Lockdown (open source) don't have this prompt and don't suffer from the Split Tunnel bug we discussed before. I hope it's just a matter of hosting the VPN on localhost.

Many thanks

ameshkov commented 3 years ago

Have you tested if Split tunnel mode actually works with "Personal VPN" apps? I tried very hard to get it to work with my WireGuard VPN, but I could only choose one or the other, not both. I may be misunderstanding how it works, for which I apologize.

It works just okay, but note that WireGuard is not a "Personal VPN", it also uses NEPacketTunnel which makes it incompatible with Full Tunenl.

In AdGuard VPN we have an additional "Integrated" mode that switches it to "Personal VPN" mode (essentially, this is IPSec) and makes it compatible with AG.

However, as I mentioned before, I get constant prompts to allow each app to connect to network devices.

Could you please show me how this prompt looks like?

ViRb3 commented 3 years ago

Ah, thank you for the explanation! Regarding the prompt, this is what I get on every internet enabled app with Full tunnel:

ameshkov commented 3 years ago

@ViRb3 hm, does it happen several times for every app or just once per app?

ViRb3 commented 3 years ago

@ameshkov It happens only once per app, but for every single app that uses the internet. It becomes hard to tell if the app actually wants to connect to LAN or it's just the Full tunnel causing it.

ameshkov commented 3 years ago

@ViRb3 I've filed an issue about that: https://github.com/AdguardTeam/AdguardForiOS/issues/1768

We'll see what can be done. Regarding Lockdown, they're running a proxy and not a DNS server for blocking stuff. We'd prefer to keep using DNS. On the other hand, if Apple does not solve this issue, we may have to run a simple proxy that'll resolve domains using the proper DNS server.

ameshkov commented 3 years ago

Good news, the original issue seems to be fixed in iOS 14.7

ViRb3 commented 3 years ago

Confirming fixed in iOS 14.7! 🎉

ameshkov commented 3 years ago

Awesome! It should be okay in iOS 15 beta as well.