uBlockOrigin / uBlock-issues

This is the community-maintained issue tracker for uBlock Origin
https://github.com/gorhill/uBlock
883 stars 76 forks source link

Beacon sending lesson data on close blocked by EasyPrivacy to uBO Trusted Site #2453

Open PinkDuck opened 1 year ago

PinkDuck commented 1 year ago

Prerequisites

I tried to reproduce the issue when...

Description

Student lesson data in a commercial system is being blocked by uBO when navigating away from an active lesson. The system first attempts a fetch() request, which fails in NS_BINDING_FAILURE then falls back to sendBeacon mechanism using text/plain JSON content. However, the async delay in sending beacon means the host can be a non-Trusted Site by the time it gets intercepted by uBO. So either the EasyPrivacy list needs an exception for this particular site, or uBO could be adjusted to allow ping requests sent to Trusted Sites when processing privacy lists.

Example blocked item extract from uBO Logger:

Filter  *$ping,3p
Filter list EasyPrivacy
 
Context www.edp24.co.uk
Partyness   (3) edp24.co.uk ⇒ ljcreatelms.com
Type    beacon
URL https://www.ljcreatelms.com/ScormData.aspx/Terminate

Static workaround: @@||www.ljcreatelms.com/ScormData.aspx/Terminate$ping

Where www.ljcreatelms.com host is a uBO Trusted Site

URL(s) where the issue occurs.

https://www.ljcreatelms.com (uBlock disabled) sendBeacon after navigate to https://www.edp24.co.uk/news (uBlock enabled)

Also https://www.ljcreatelms.co.uk/ and https://lp1.ljcreatelms.com/

Screenshot(s)

image

uBO version

1.46.0

Browser name and version

Mozilla Firefox x64 108.0.2

Settings

Added "www.ljcreatelms.com" (without quotes) to Trusted Sites, confirmed uBO inactive at that site

Notes

Tried disabling uBO in Firefox Extensions, issue did not occur Tried navigating to another trusted site, issue did not occur Only navigating from trusted to untrusted blocks the Beacon being sent back to Trusted Clearing Firefox Cache (Everything) seems to increase the likelihood of occurrence Not an issue using either AdBlock or ABP with respective EasyPrivacy/tracker options Noticed Firefox Fetch API keepalive support bug and direct use of Beacon during beforeunload succeeds

mapx- commented 1 year ago

@gorhill

gwarser commented 1 year ago

This works also in the other way around.

  1. Whitelist https://example.com/
  2. search ddg for it https://duckduckgo.com/?q=https%3A%2F%2Fexample.com%2F&ia=web
  3. there will be a direct link to example.com in wiki widget on the right
  4. click on it
  5. uBO does not block ddg beacon neither by ||duckduckgo.com/t/ or *$ping,3p because it's called from whitelisted example.com context.

See also: https://github.com/uBlockOrigin/uBlock-issues/issues/2363

gwarser commented 1 year ago

Still not blocked when navigating to whitelisted page.

Still blocked when navigating from whitelisted page.

Trying with:

||duckduckgo.com/t/$badfilter
*$ping,3p,badfilter
*$ping,strict3p

Because looks like *$ping,3p is correctly not applied now. On example.com beacon has duckduckgo.com context, so is 1p.

Similarly, with *$ping,1p.

PinkDuck commented 1 year ago

A key issue can be seen at CanIUse/keepalive in that some user agents (Firefox included) claim Fetch API full support, but actually a Fetch during beforeunload/unload does not have guaranteed network delivery and can be seen failing with NS_BINDING_ABORT. So I added user agent detection to compensate by using Beacon API directly, the first of these which does not get blocked, though the second ping does oddly.

gorhill commented 1 year ago

Still not blocked when navigating to whitelisted page.

I am aware, this is another issue in a different code path, not as easily fixed.


The difficulty with the trusted-site scenario is that disabling uBO is a test against the top-level context, and by the time the beacon is processed by uBO, the top-level context has changed.


This is the details I get re. ping/beacon requests (in chronological order) in Chromium/Firefox when navigating from duckduckgo.com to example.com, so this is the information uBO has to deal with:

Chromium ```json { "documentId": "E015EE64722A8657E9F4A36AEC96DD17", "documentLifecycle": "active", "frameId": 0, "frameType": "outermost_frame", "initiator": "https://duckduckgo.com", "method": "POST", "parentFrameId": -1, "requestId": "1363", "tabId": 207876891, "timeStamp": 1675004947732.364, "type": "ping", "url": "https://improving.duckduckgo.com/t/pae?2236082&q=https%3A%2F%2Fexample.com%2F&ttc=70065&ct=CA&d=d&serp_return=1&g=__&sm=server_fault:q:low&bkl=r1-8&blay=w30r1,e1&dsig=nlp_qa:m&be_eclsexp=b&be_msvrtexp=b&biaexp=b&cproxexp=b&eclsexp=b&euctaexp=b&infoboxexp=b&litexp=a&msvrtexp=b&mxmexp=a&videxp=a", "documentUrl": "https://duckduckgo.com" } { "documentId": "E015EE64722A8657E9F4A36AEC96DD17", "documentLifecycle": "active", "frameId": 0, "frameType": "outermost_frame", "initiator": "https://duckduckgo.com", "method": "POST", "parentFrameId": -1, "requestId": "1364", "tabId": 207876891, "timeStamp": 1675004947735.7732, "type": "ping", "url": "https://improving.duckduckgo.com/t/lc?5629835&t=d&ss=0&sp=0&osl=6&dm=example.com&hn=example.com&u=bingv7aa&nt=0&r=r8&da=0&rl=us-en&dl=en&oll=en:24,es:1,ko:1,pl:1,zh-CN:1&pr=http&i506=0&ivc=1&q=https%3A%2F%2Fexample.com%2F&ttc=70069&ct=CA&d=d&kl=wt-wt&kp=-1&serp_return=1&g=__&sm=server_fault:q:low&bkl=r1-8&blay=w30r1,e1&dsig=nlp_qa:m&be_eclsexp=b&be_msvrtexp=b&biaexp=b&cproxexp=b&eclsexp=b&euctaexp=b&infoboxexp=b&litexp=a&msvrtexp=b&mxmexp=a&videxp=a", "documentUrl": "https://duckduckgo.com" } { "documentLifecycle": "active", "frameId": 0, "frameType": "outermost_frame", "initiator": "https://duckduckgo.com", "method": "GET", "parentFrameId": -1, "requestId": "1366", "tabId": 207876891, "timeStamp": 1675004947749.509, "type": "main_frame", "url": "http://example.com/", "documentUrl": "https://duckduckgo.com" } { "documentId": "E015EE64722A8657E9F4A36AEC96DD17", "documentLifecycle": "cached", "frameId": 20, "frameType": "outermost_frame", "initiator": "https://duckduckgo.com", "method": "POST", "parentFrameId": -1, "requestId": "1367", "tabId": 207876891, "timeStamp": 1675004947771.949, "type": "ping", "url": "https://improving.duckduckgo.com/t/webvitals?8113009&FCP=175&TTFB=24&LCP=329&FID=29&CLS=0.1847&effectiveType=4g&has_performance=1&is_cached=0&navigation_type=back_forward&has_back_data=1&is_loaded_from_bfcache=0&is_bounce_back=1&g=__&sm=server_fault:q:low&bkl=r1-8&blay=w30r1,e1&dsig=nlp_qa:m&be_eclsexp=b&be_msvrtexp=b&biaexp=b&cproxexp=b&eclsexp=b&euctaexp=b&infoboxexp=b&litexp=a&msvrtexp=b&mxmexp=a&videxp=a", "documentUrl": "https://duckduckgo.com" } ```
Firefox ```json { "requestId": "282", "url": "https://improving.duckduckgo.com/t/pae?2455971&q=https%3A%2F…nt_single_place=a&factexp=a&infoboxexp=e&litexp=c&msvrtexp=b", "originUrl": "https://duckduckgo.com/?q=https%3A%2F%2Fexample.com%2F&ia=web", "documentUrl": "https://duckduckgo.com/?q=https%3A%2F%2Fexample.com%2F&ia=web", "method": "POST", "type": "beacon", "timeStamp": 1675005871720, "tabId": 6, "frameId": 0, "parentFrameId": -1, "incognito": false, "thirdParty": false, "cookieStoreId": "firefox-default", "proxyInfo": null, "ip": null, "frameAncestors": [], "urlClassification": { "firstParty": [], "thirdParty": [] }, "requestSize": 0, "responseSize": 0 } { "requestId": "284", "url": "https://improving.duckduckgo.com/t/lc?6232704&t=d&ss=0&sp=0&…nt_single_place=a&factexp=a&infoboxexp=e&litexp=c&msvrtexp=b", "originUrl": "https://duckduckgo.com/?q=https%3A%2F%2Fexample.com%2F&ia=web", "documentUrl": "https://duckduckgo.com/?q=https%3A%2F%2Fexample.com%2F&ia=web", "method": "POST", "type": "beacon", "timeStamp": 1675005871733, "tabId": 6, "frameId": 0, "parentFrameId": -1, "incognito": false, "thirdParty": false, "cookieStoreId": "firefox-default", "proxyInfo": null, "ip": null, "frameAncestors": [], "urlClassification": { "firstParty": [], "thirdParty": [] }, "requestSize": 0, "responseSize": 0 } { "requestId": "285", "url": "http://example.com/", "originUrl": "https://duckduckgo.com/?q=https%3A%2F%2Fexample.com%2F&ia=web", "method": "GET", "type": "main_frame", "timeStamp": 1675005871734, "tabId": 6, "frameId": 0, "parentFrameId": -1, "incognito": false, "thirdParty": false, "cookieStoreId": "firefox-default", "proxyInfo": null, "ip": null, "frameAncestors": [], "urlClassification": { "firstParty": [], "thirdParty": [] }, "requestSize": 0, "responseSize": 0 } { "requestId": "286", "url": "https://improving.duckduckgo.com/t/webvitals?9334703&FCP=365…nt_single_place=a&factexp=a&infoboxexp=e&litexp=c&msvrtexp=b", "originUrl": "https://duckduckgo.com/?q=https%3A%2F%2Fexample.com%2F&ia=web", "documentUrl": "https://duckduckgo.com/?q=https%3A%2F%2Fexample.com%2F&ia=web", "method": "POST", "type": "beacon", "timeStamp": 1675005871755, "tabId": 6, "frameId": 0, "parentFrameId": -1, "incognito": false, "thirdParty": false, "cookieStoreId": "firefox-default", "proxyInfo": null, "ip": null, "frameAncestors": [], "urlClassification": { "firstParty": [], "thirdParty": [] }, "requestSize": 0, "responseSize": 0 } ```