Whitelist does not work for requests sent from within a service worker

jkldgoefgkljefogeg commented 6 years ago

Steps to reproduce

1.open https://www.facebook.com/ (logged in) 2.turn off "protection on this website" 3.open https://www.facebook.com/settings 4.click on ads at bottom of left navigation bar

Expected behavior

no ad blocking running

Actual behavior

page returns This site can’t be reached

console The service worker navigation preload request failed with network error: net::ERR_BLOCKED_BY_CLIENT. The FetchEvent for "https://www.facebook.com/ads/preferences/?entry_product=ad_settings_screen" resulted in a network error response: the promise was rejected

Screenshot:

![adguard_switch_not_working](https://user-images.githubusercontent.com/1191627/40883326-b2093862-66ae-11e8-8b51-788d40e970e7.png)

Screenshot of the browser console:

Your environment

AdGuard AdBlocker extension version: 2.9.2
Environment name and version: (e.g. Chrome 59.0.3071.115) Chrome stable Google Chrome | 67.0.3396.62 (Official Build) (64-bit)
Filters you use in AdGuard: Russian filterVersion 2.0.3.56. Updated 1 june 2018 at 14:02. German filterVersion 2.0.1.71. Updated 1 june 2018 at 17:03. French filterVersion 2.0.3.76. Updated 1 june 2018 at 20:02. Dutch filterVersion 2.0.0.30. Updated 29 may 2018 at 14:12. EasyList ChinaVersion 2.0.9.25. Updated 1 june 2018 at 23:01. EasyList Czech and SlovakVersion 2.0.0.38. Updated 2 june 2018 at 05:02.
Any other browser extensions you have installed: Lastpass, HTTPS Everywhere

I have previously ran into other sites where whitelist was not working and I had to disable the extension for the site to load properly.

maximtop commented 6 years ago

@jkldgoefgkljefogeg @ameshkov hi, I couldn't reproduce the issue. I tried on my own chrome profile, and on the new one with listed filters, adding Facebook to the whitelist. @jkldgoefgkljefogeg Could you check please if you can reproduce the issue with new chrome profile?

jkldgoefgkljefogeg commented 6 years ago

I can reproduce it with a different profile on

Google Chrome | 67.0.3396.62 (Official Build) (64-bit) Revision | babbbb5b433370f9a7feeb9f98a57599ad1c4676-refs/branch-heads/3396@{#702} OS | Linux

also on Google Chrome 66.0.3359.181 (Official Build) (64-bit) Revision a10b9cedb40738cb152f8148ddab4891df876959-refs/branch-heads/3359@{#828} OS Mac OS X

With www.facebook.com in ADguard whitelist, anything with https://www.facebook.com/ads/ or https://www.facebook.com/advertising/ fails with

The service worker navigation preload request failed with network error: net::ERR_BLOCKED_BY_CLIENT.

maximtop commented 6 years ago

@jkldgoefgkljefogeg Do you have any user rules related to facebook?

jkldgoefgkljefogeg commented 6 years ago

No user filter in all 3 cases (2 different profiles on Linux, 1 on Mac)

maximtop commented 6 years ago

Ok, let's try another way to whitelist facebook. Could you please remove the site from the whitelist, and add the rule @@||facebook.com to the user rules list. Has it helped? If not send please all requests from the console

jkldgoefgkljefogeg commented 6 years ago

@@||facebook.com turned into %7C%7Cfacebook.com on submission. Same result

%7C%7Cfacebook.com does not even disable adblocking on https://www.facebook.com/

maximtop commented 6 years ago

@jkldgoefgkljefogeg Insert this rule please in the section of user rules Look at the attached screenshot, please

jkldgoefgkljefogeg commented 6 years ago

With @@||facebook.com user filter, facebook.com is still showing green and ads blocked, but https://www.facebook.com/ads/ and https://www.facebook.com/advertising/ can load successfully

maximtop commented 6 years ago

The rule @@||facebook.com^$document should completely disable add blocking for facebook Anyway, we will try to figure out why whitelist is not working in your case. Could you send please network tab from the console with requests, when the page can't be reached? For example, see screenshot.

jkldgoefgkljefogeg commented 6 years ago

net-internals events

6936: URL_REQUEST
https://www.facebook.com/ads/create/?campaign_id=276892347393432&placement=tcr&extra_1=not-admgr-user
Start Time: 2018-06-05 01:22:18.296

t=6567 [st= 0] +REQUEST_ALIVE  [dt=11]
                --> priority = "HIGHEST"
                --> url = "https://www.facebook.com/ads/create/?campaign_id=276892347393432&placement=tcr&extra_1=not-admgr-user"
t=6568 [st= 1]   +URL_REQUEST_DELEGATE  [dt=2]
t=6568 [st= 1]      DELEGATE_INFO  [dt=2]
                    --> delegate_blocked_by = "extension AdGuard AdBlocker"
t=6570 [st= 3]   -URL_REQUEST_DELEGATE
t=6571 [st= 4]   +URL_REQUEST_START_JOB  [dt=7]
                  --> load_flags = 18434 (BYPASS_CACHE | MAIN_FRAME_DEPRECATED | MAYBE_USER_GESTURE)
                  --> method = "GET"
                  --> url = "https://www.facebook.com/ads/create/?campaign_id=276892347393432&placement=tcr&extra_1=not-admgr-user"
t=6571 [st= 4]      SERVICE_WORKER_START_REQUEST
t=6571 [st= 4]     +SERVICE_WORKER_DISPATCH_FETCH_EVENT  [dt=7]
                    --> event_type = "Fetch Main Frame"
t=6572 [st= 5]        SERVICE_WORKER_FETCH_EVENT  [dt=6]
t=6578 [st=11]     -SERVICE_WORKER_DISPATCH_FETCH_EVENT
                    --> has_response = true
                    --> status = "Operation has succeeded"
t=6578 [st=11]      SERVICE_WORKER_ERROR_RESPONSE_STATUS_ZERO
t=6578 [st=11]   -URL_REQUEST_START_JOB
                  --> net_error = -2 (ERR_FAILED)
t=6578 [st=11]    URL_REQUEST_DELEGATE  [dt=0]
t=6578 [st=11] -REQUEST_ALIVE
                --> net_error = -2 (ERR_FAILED)

>
6937: URL_REQUEST
https://www.facebook.com/ads/create/?campaign_id=276892347393432&placement=tcr&extra_1=not-admgr-user
Start Time: 2018-06-05 01:22:18.300

t=6571 [st=0] +REQUEST_ALIVE  [dt=6]
               --> priority = "HIGHEST"
               --> url = "https://www.facebook.com/ads/create/?campaign_id=276892347393432&placement=tcr&extra_1=not-admgr-user"
t=6571 [st=0]    SERVICE_WORKER_NAVIGATION_PRELOAD_REQUEST
t=6572 [st=1]   +URL_REQUEST_DELEGATE  [dt=4]
t=6572 [st=1]      DELEGATE_INFO  [dt=4]
                   --> delegate_blocked_by = "extension AdGuard AdBlocker"
t=6576 [st=5]      CHROME_EXTENSION_ABORTED_REQUEST
                   --> extension_id = "bgnkhhnnamicmpeenaelnjfhikgbkllg"
t=6576 [st=5]   -URL_REQUEST_DELEGATE
t=6576 [st=5]    CANCELLED
                 --> source = "delegate"
t=6576 [st=5]   +URL_REQUEST_START_JOB  [dt=0]
                 --> load_flags = 18434 (BYPASS_CACHE | MAIN_FRAME_DEPRECATED | MAYBE_USER_GESTURE)
                 --> method = "GET"
                 --> url = "https://www.facebook.com/ads/create/?campaign_id=276892347393432&placement=tcr&extra_1=not-admgr-user"
t=6576 [st=5]   -URL_REQUEST_START_JOB
                 --> net_error = -20 (ERR_BLOCKED_BY_CLIENT)
t=6577 [st=6]    URL_REQUEST_DELEGATE  [dt=0]
t=6577 [st=6] -REQUEST_ALIVE
               --> net_error = -20 (ERR_BLOCKED_BY_CLIENT)

adguard_whitelist_bug_chrome_dev_console_network

maximtop commented 6 years ago

Seems like requests are sent by server workers. Server workers don't have tabId. That's why the whitelisted domain doesn't work. We can add a rule in the base filter list which would allow access to the facebook.com/ads/create pages. What do you think, would it be enough for your needs?

jkldgoefgkljefogeg commented 6 years ago

Is something like this feasible in Adguard? hardcoding URL seems problematic.

ameshkov commented 6 years ago

Hm, this is a good idea indeed!

ameshkov commented 6 years ago

@maximtop besides using originUrl/initiator while matching the request, you need to pass it to the filtering log as well.

maximtop commented 6 years ago

In order to check if this issue was fixed, you should

install AdGuard Extension v2.10
open https://regex101.com/
add rule ||regex101.com/sw.js$domain=regex101.com
check if the request to sw.js was blocked
disable protection for the current site
check if the request to sw.js is not blocked

AdguardTeam / AdguardBrowserExtension