search

AdguardTeam / AdguardForAndroid

Open bug tracker for Android version of AdGuard.
https://adguard.com/
1.37k stars 90 forks source link

unable to create custom firewall rules to allow download of android updates from google #5327

Open DonEstefan opened 6 months ago

DonEstefan commented 6 months ago

Please answer the following questions for yourself before submitting an issue

AdGuard version

4.3.1 + 4.4

Environment

HTTPS filtering

Root access

Integration with AdGuard VPN

Routing mode

Local VPN

Ad Blocking

AdGuard Base filter, AdGuard Mobile Ads filter, EasyList

Privacy

AdGuard Tracking Protection filter, EasyPrivacy

Social

AdGuard Social Media filter

Annoyances

AdGuard Popups filter, AdGuard Mobile App Banners filter

Security

Online Malicious URL Blocklist

Language-specific

AdGuard Japanese filter, EasyList Germany

Other

No response

Which DNS server do you use?

Automatic DNS

DNS protocol

None

Custom DNS

No response

What Stealth Mode options do you have enabled?

No response

Issue Details

Steps to reproduce:

  1. enable adguard (DNS+Firewall) .
  2. enable global firewall rules in adguard. disallow all wifi + mobile data connections globally.
  3. go to android settings and try to download available android security updates provided by google. This results in the error message "network error. try again later" seen in the german screenshot below
  4. check adguard firewall quick actions -> no quick action available, to unlock the android updates
  5. check "recent activity" in adguard "statistics" tab-> note the blocked tcp connection from "root" in the second screenshot below. However, there is no app called "root" installed on my phone, so I can not allow this connection/app...
  6. enable "custom firewall rule". Allow all wifi+mobile data for all "Download manager" and "system" apps. Repeat step 3 -> Download still does not work
  7. disable adguard and repeat step 3 -> android update download works as expected

Expected Behavior

There should be a way to build custom firewall rules for android updates, when global firewall rules do not allow internet access by default.

Actual Behavior

android update downloads always fails, when global firewall rules do not allow internet access by default

Screenshots

Android Update error (german) ![grafik](https://github.com/AdguardTeam/AdguardForAndroid/assets/8457107/9ea3585e-6a46-445c-a743-3760124e4682)
I suspect this is the blocked update download ![grafik](https://github.com/AdguardTeam/AdguardForAndroid/assets/8457107/f4084bea-c8e4-4d69-a245-dd6c285bda86)

Additional Information

No response

maxikuzmin commented 6 months ago

@DonEstefan

try to download available android security updates provided by google

could you please clarify the path to this setting? you mean Settings -> Security & Privacy -> System & updates -> Security update?

DonEstefan commented 6 months ago

could you please clarify the path to this setting? you mean Settings -> Security & Privacy -> System & updates -> Security update?

Correct

maxikuzmin commented 6 months ago

@DonEstefan well, I tested this on Pixel 5 (android 14) and everything works correctly

Please watch the video, are you doing the same steps?

https://github.com/AdguardTeam/AdguardForAndroid/assets/101641415/87581fec-0a6e-4cb1-bd8f-571e0ee1ac5f

As far as I understand the blocked request is called "google play services", I don't have any requests from "root" showing up, and I was able to temporarily disable Firewall for the update and updated successfully

I also checked the Security update on another Samsung device (android 13), but everything works correctly there as well. Do you have a chance to check this on another device, will it also recur? Most likely the problem is in your device

maxikuzmin commented 6 months ago

@DonEstefan by the way about "root" If the system makes requests with UID 0, then a request from a process with UID 0 is described as "root". It's normal behavior

DonEstefan commented 6 months ago

Hi @maxikuzmin. And thanks for your efforts! You are looking at the right setting. But the check for updates button is working fine ("play services" are allowed to access internet on my phone). It's the download and install button that came after check for updates, which did not work. Unfortunately I can not create a screen recording of the issue, since I installed all available android updates during my earlier tests.

maxikuzmin commented 6 months ago

@DonEstefan It's really hard to check this, since Security updates are released very rarely (I had my last update in November, and it's the "most recent" update available

DonEstefan commented 6 months ago

@maxikuzmin, I get monthly updates with my pixel phone. Let me know, if I can help...

maxikuzmin commented 5 months ago

@DonEstefan okay, so the problem is: Quick Actions for the Adguard Firewall are not available when blocking Security update downloads After disabling the Firewall, downloading updates still doesn't happen until you turn off AdGuard protection completely

Is this correct? Unfortunately, my updates are still showing as “Your system is up to date” from Nov 05, maybe the difference is in the firmware of our devices.

Could you please record a video of the screen repeating all the steps and also record debug logs?

Settings -> General -> Advanced -> Logging level -> Debug

Settings -> General -> Advanced -> Export logs and system info

mention the 5327 number in the subject specify the exact time when the issue occurred

maxikuzmin commented 5 months ago

@DonEstefan any news?

DonEstefan commented 5 months ago

@maxikuzmin, I'm on a festival right now. There are new updates available on my phone. I'll get the debug los for you, when I'm back next week.

maxikuzmin commented 5 months ago

@DonEstefan okay, I'll be waiting. Have fun!

DonEstefan commented 5 months ago

@maxikuzmin, I send the file you asked for to the devteam mailadress. Let me know if you need additional info.

maxikuzmin commented 5 months ago

@DonEstefan could you record a new video of the screen? Important, when you open a request in recent activity, scroll below to see if the request was blocked by a firewall

maxikuzmin commented 5 months ago

@DonEstefan any news?

DonEstefan commented 5 months ago

Hi @maxikuzmin, I sent you another screen recording. I hope it helps. I'm abroad for the next 2 weeks, so I use a roaming connection. But it should not make much difference. Do you have an idea how to unblock the logged connections created by "root"?

maxikuzmin commented 4 months ago

@DonEstefan to unblock any request, you can tap on the request in Recent activity, then scroll down and tap on "Add allowing rule"

Thanks for the video, we'll look into it

maxikuzmin commented 4 months ago

@DonEstefan well, I watched your screen video, but you again just opened requests but didn't scroll down :) Please do as I asked you before: do the same steps but when opening a request in Recent activity scroll down

We need to see what is displayed at the bottom of the request

DonEstefan commented 4 months ago

Hi @maxikuzmin. Sorry, for the misunderstanding. At the very bottom of the "request details" of the blocked "root" connections there is just 1 additional line saying no applied rules. There is no "add allowing rule" option when clicking on that line (or anywhere else on the connection detail screen). I attached a Foto. Do you still need an additional video? I'm up in the mountains and it might need some time until I have proper internet to upload video. Screenshot_20240705-230713

maxikuzmin commented 4 months ago

@DonEstefan we will need some more time, please wait

maxikuzmin commented 4 months ago

@DonEstefan we need logs with a newer version of AdGuard (version 4.5). Could you upload the logs with the newer version again please?

DonEstefan commented 4 months ago

@maxikuzmin, I sent a download link for the logs to the devteam mail address.

maxikuzmin commented 4 months ago

@DonEstefan I don't see any new emails from you in the mail, please resend email

maxikuzmin commented 4 months ago

@DonEstefan we need some more time. Please wait

maxikuzmin commented 3 months ago

@DonEstefan so, a failed update attempt can be for various reasons: DNS may be blocking the requests, filter rules may be affecting in some way, system applications may be blocking. In this case it would be better to reset AdGuard to default values and try again

DonEstefan commented 3 months ago

@maxikuzmin, we'll have to wait another week or so before I can do more testing. Unfortunately, there are no android updates available right now. But I really don't think the problem is related to DNS blocking or filter rules from the ad_blocking/annoyance_blocking protection modules of adguard - but only related to the "Firewall" module of adguard. For me, the obvious reason its blocked is because all connections on my phone are blocked through the "global firewall rules" (see screenshot below). If I want an app to access the internet, then I need to manually create an "custom firewall rule" for that app - which is currently not possible for the app/user shown as "root" in the logs. This assumption would explain the behavior we see and also the log entries we see. It sounds like you don't agree with this assumption?

grafik

DonEstefan commented 2 months ago

Today I disabled all filter modules, except for the firewall module. The behavior is still the same. Android updates are blocked, log shows blocked connections for "root" and no way to allow these. Adguard is running in version 4.6.2 now.

DonEstefan commented 2 months ago

@maxikuzmin, I did some more tests using the Low-Level Settings of Adguard.

  1. I tried adding "root" and "0" to the "excluded apps" in the low-level settings. However, this did not make the updates work. I still faced the same problem.

  2. I added the IPv4 Addresses of the servers used by Pixel devices for OTA updates to "IPv4 ranges excluded from Filtering". This actually worked and solved the download problem. (ota.googlezip.net=216.239.32.112, ota-cache1.googlezip.net=209.85.137.4, ota-cache2.googlezip.net=209.85.137.5)

    //Whitelist Google Pixel OTA update servers from https://support.google.com/work/android/answer/10513641
216.239.32.112/32
209.85.137.4/32
209.85.137.5/32

  3. I added the IPv6 Addresses of the Google Pixel OTA update servers to "IPv6 ranges excluded from Filtering". This also worked and solved the download problem.

    //Whitelist Google Pixel OTA update servers from https://support.google.com/work/android/answer/10513641
2001:4860:4802:32::70/128
2001:4860:3::4/128
2001:4860:3::5/128

I hope there are smarter ways to unblock the connections, since the IP's might change anytime. And of course an adguard firewall rule would generally be better suited, than an low-level adguard bypass. What do you think about this?

maxikuzmin commented 2 months ago

@DonEstefan do you still have Download manager and system excluded from firewall?

DonEstefan commented 2 months ago

@maxikuzmin, both app groups have a firewall rule allowing access. This is how it looks:

Screenshot_20240908-141225

maxikuzmin commented 2 months ago

@DonEstefan we have investigated this problem: appsProvider does not give root request, hence a rule for it unfortunately cannot be made. Because of this limitation it is impossible to make a rule for it

DonEstefan commented 2 months ago

@maxikuzmin, thanks for the investigation. Just for my understanding, is "appsProvider" and android component or an adguard component?

maxikuzmin commented 3 weeks ago

@DonEstefan this logic will be improved in the next updates, we'll fix it, thank you