EFForg / privacybadger

Privacy Badger is a browser extension that automatically learns to block invisible trackers.
https://privacybadger.org
Other
3.19k stars 386 forks source link

Privacy Badger breaks some deliberate third party interactions such as oAuth with facebook youtube disqus etc. #137

Open cooperq opened 10 years ago

cooperq commented 10 years ago

Sometimes when a user tries to use oAuth to log into a site with facebook or google or tries to use disqus to comment on a site or makes some other deliberate interaction with a third party resource it gets blocked. This presents a bad experience to the user who is prevented from using a website in the way they intended. We need to find a generalizable solution to this

Current Workarounds

facebook oauth and comments: unblock www.facebook.com unblock connect.facebook.com unblock static.ak.facebook.com unblock s-static.ak.facebook.com

you tube comments: unblock apis.google.com

disqus comments: unblock disqus.com

google oauth: unblock oauth.googleusercontent.com unblock apis.google.com

cooperq commented 10 years ago

This is a similar enough problem to #147 and #142 that I think they can all be solved the same way. Here is the solution as I see it:

User Story

Alice visits youtube.com and clicks on the text box below a video to comment. If the necesary domain (in this case apis.google.com) is not already allowed then Alice gets a popup telling her that she needs to allow this domain to take this action, but by doing that she could be letting the domain track her browsing. If Alice clicks 'OK' then the domain is put into Alices white list and Alice makes a comment on youtube. If Alice clicks 'Cancel' then the domain stays as it is and Alice continues on with her day. If Alice trys to comment on youtube again and it is still blocked, then she will once again get the popup.

Pseudo Code

OnBeforeRequest() do
  if requestDomain in exceptionDomains 
  and request domain would be blocked or cookieblocked do
    displayPopup('the action you are taking would need to allow this domain to track you. Do you want that to happen?')
    if yes then move domain to user whitelist
    if no then do nothing
  done
done

/* This is a dictionary containing all of the domains that user may want to make an exception for. The key is the domain that will need to be allowed and the value is an array of the acceptable 1st party domains to show this popup on. If the array is empty then we show the popup for any request to the domain */
exceptionDomains = {
  'login.disqus.com': [],
  'apis.google.com': ['youtube.com'],
}
cooperq commented 10 years ago

One argument against any of this is that the user could already do this by just whitelisting the appropriate sites, or the sites could whitelist themselves by posting dnt-policy.txt and that having this feature rewards bigger content providers by making it easier for the user to unblock them. It also puts us in the position of maintaining two whitelists. An alternate proposal is that we just do this for every site on the cookie block list and then we only have to maintain one list of domains.

cooperq commented 10 years ago

Now that I think about this more I don't think that there is any need to have a list of referrers that we care about for each domain. The exception domains could just be an array then, which has the added bonus of giving us a performance boost.

exceptionDomains = [
  'https://apis.google.com/u/0/wm/4/_/diagnostics/',                            
  'https://disqus.com/next/login/',  
]
cooperq commented 10 years ago

Additionally we should only bring up this dialog if privacy badger has automatically blocked these domains. If the user has manually blocked these domains then we should trust their judgement and not bring up this dialog IMO.

yawpitch commented 10 years ago

It also breaks Facebook login on the the Kinja-based sits over at Gawker. Workaround is to:

unblock .facebook.com unblock .kinja.com unblock stats.g.doubleclick.net

The last one surprises me, but without it also being deactivated login fails entirely.

deanishe commented 10 years ago

Instapaper is still broken :cry:

The PB dialog pops up asking whether and when to allow instapaper.com, but the browser is redirected to the instapaper.com website too fast to let you click a dialog button.

How about a nice, simple text box in the settings where I can add any domains I'd like to whitelist?

cooperq commented 10 years ago

It is annoying that instapaper does that redirect, I wonder if there is some way to prevent that. I agree that there should be a way to whitelist domains, but text boxes aren't super user friendly. I think that the better way to fix this is to have an options page for privacy badger that shows you all of the domains that it knows about and lets you override the settings for each of them.

deanishe commented 10 years ago

The Instapaper bookmarklet does the redirect because PB is blocking it. The alternative is not working at all. If PB prevented the redirect, it would completely break the bookmarklet.

A "proper" interface would be better than a text box, that is certainly true.

ghostwords commented 6 years ago

Reopening as #219 has since been removed from Privacy Badger (as part of #951, I think).

From a recent AMO user review:

[...] disables logging in with google, facebook and other IDs on some websites. [...] For example, it is not possible to use draw.io services if this addon is enabled - integrations with google are not working and privacy badger closes all windows that attempt to authenticate with google services. Similar with Shazam - not possible to login with FB.

ghostwords commented 6 years ago

We've been getting a number of related complaints lately regarding not being able to authenticate through your cable provider to watch the Olympics. I think they all have to do with Privacy Badger "cookieblocking" auth.adobe.com (1fda4d93fbc4d0ce4b60f8be48fac22e1a7679f7), which breaks the authentication flow.

To be clear, auth.adobe.com being on the yellowlist is not the problem (blocking it would also break the flow); it's just that the yellowlist is not the right tool to work around site breakages where sites require third-party cookies/localStorage to function.

Some sample reports from 2018.2.5 (latest version) users: ``` url: http://www.espn.com/watch/?id=3238901 block: c.amazon-adsystem.com,adservice.google.com,securepubads.g.doubleclick.net,cdn-gl.imrworldwide.com,b.scorecardresearch.com cookieblock: cdn.optimizely.com,310987714.log.optimizely.com,logx.optimizely.com,entitlement.auth.adobe.com,sp.auth.adobe.com noaction: www.googletagservices.com message: www.espn.com/watch does not work with the badger. :( It does not allow verification ``` ``` url: http://stream.nbcolympics.com/curling-mixed-doubles-round-robin-sheet-a-am-session-day-minus-1 block: c.betrad.com,dpm.demdex.net,adservice.google.com,securepubads.g.doubleclick.net,connect.facebook.net,b.scorecardresearch.com,js-agent.newrelic.com,fast.nbcu.demdex.net,nbcume.sc.omtrdc.net,nbcu.demdex.net cookieblock: mps.nbcuni.com,www.googletagservices.com,entitlement.auth.adobe.com,sp.auth.adobe.com message: Doesn't seem to handle authentication for NBC winter olympics coverage. ``` ``` url: http://www.msnbc.com/now block: cdn.dynamicyield.com,st.dynamicyield.com,static.dynamicyield.com,adservice.google.com,securepubads.g.doubleclick.net,cdn.krxd.net,contributor.google.com,cdn-akamai.mookie1.com,static.parsely.com,secure-us.imrworldwide.com,dpm.demdex.net,px.dynamicyield.com,b.scorecardresearch.com,nbcu.demdex.net,srv-2018-02-13-01.config.parsely.com,pagead2.googlesyndication.com,tpc.googlesyndication.com,secure-ds.serving-sys.com,srv-2018-02-13-01.pixel.parsely.com,t.mookie1.com,us-gmtdmp.mookie1.com,usermatch.krxd.net,cms.analytics.yahoo.com,idsync.rlcdn.com,sync-tm.everesttech.net,pixel.advertising.com,match.adsrvr.org,tags.bluekai.com,bea4.v.fwmrm.net,pixel.mathtag.com,pixel.quantserve.com,beacon.krxd.net,r.nexac.com,aa.agkn.com,image2.pubmatic.com,www.googleadservices.com,secure.adnxs.com,bs.serving-sys.com,sb.scorecardresearch.com,r.dlx.addthis.com,d.agkn.com,googleads.g.doubleclick.net,geo-um.btrll.com,cache.btrll.com,fast.nbcuni.demdex.net,fast.nbcu.demdex.net,secure.quantserve.com,cm.everesttech.net,pixel.everesttech.net,loadm.exelator.com,su.addthis.com,rrc.rlcdn.com,load77.exelator.com,cm.g.doubleclick.net,dmp.v.fwmrm.net,mid.rkdms.com,ads.yahoo.com,rtd-tm.everesttech.net,image5.pubmatic.com,ssum.casalemedia.com,cookiex.ngd.yahoo.com,ps.eyeota.net,ads.scorecardresearch.com,29773.v.fwmrm.net cookieblock: mps.nbcuni.com,tags.tiqcdn.com,player.theplatform.com,feed.entertainment.tv.theplatform.com,www.facebook.com,pdk.theplatform.com,tve-common.nbcuni.com,entitlement.auth.adobe.com,oimg.nbcuni.com,mvpd-admin.nbcuni.com,sp.auth.adobe.com,feed.theplatform.com,adm.fwmrm.net,www.google.com,link.theplatform.com,tve-static-msnbc.nbcuni.com,tvemsnbc-lh.akamaihd.net noaction: static.hotjar.com,cdn.mxpnl.com,script.hotjar.com,js-sec.indexww.com,api.mixpanel.com,www.googletagservices.com,media1.s-nbcnews.com,tracker.nbcuas.com,p.d.e0mn.com,ak.sail-horizon.com,www.google-analytics.com,connect.facebook.net,nervoussummer.com,vars.hotjar.com,c.betrad.com,l.betrad.com,code.jquery.com,rules.quantcount.com,js.moatads.com,players.edgesuite.net message: wouldn't let sign in from Xfinity to stream on MSNBC ``` Might be different issues: ``` url: https://www.history.com/topics/world-war-ii/world-war-ii-history/videos/the-flying-tigers-of-world-war-ii block: c.amazon-adsystem.com,ad.doubleclick.net,adservice.google.com,securepubads.g.doubleclick.net,sb.scorecardresearch.com,cdn.krxd.net,secure-us.imrworldwide.com,stats.g.doubleclick.net,bs.serving-sys.com,www.googleadservices.com,dpm.demdex.net,tag.mtrcs.samba.tv,ds.reson8.com,cdn-gl.imrworldwide.com,6765880.fls.doubleclick.net,uconnect.tealiumiq.com,6758024.fls.doubleclick.net,mssl.fwmrm.net,app.link cookieblock: cdn.optimizely.com,logx.optimizely.com,fonts.gstatic.com,entitlement.auth.adobe.com,a14322303.cdn.optimizely.com,cdn3.optimizely.com,www.googletagservices.com,tags.tiqcdn.com,sp.auth.adobe.com,a248.e.akamai.net,s3.amazonaws.com message: ad blocker detected ``` ``` url: https://www.c-span.org/video/?440920-18/federal-government-shuts-funding-expires-senate-schedules-1am-cloture-vote-restore-funding&live cookieblock: entitlement.auth.adobe.com,sp.auth.adobe.com,cspan2nontve-lh.akamaihd.net noaction: use.typekit.net,p.typekit.net,vjs.zencdn.net ``` ``` url: https://www.nbc.com/this-is-us/video/watch-this-is-us-from-the-beginning/3663257 block: cdn.betrad.com,cdn.petametrics.com,dpm.demdex.net,api.mixpanel.com,nbcume.sc.omtrdc.net,l.betrad.com,pubads.g.doubleclick.net,adservice.google.com,securepubads.g.doubleclick.net,nbcu.demdex.net,4481714.fls.doubleclick.net,sb.scorecardresearch.com,cdn.krxd.net,secure-us.imrworldwide.com,ad.doubleclick.net,tpc.googlesyndication.com cookieblock: tve-common.nbcuni.com,mps.nbcuni.com,sp.auth.adobe.com,pix.nbcuni.com noaction: www.leanplum.com,jssdks.mparticle.com,nervoussummer.com,tracker.nbcuas.com,3nkvntt7f3-dsn.algolia.net,smart.link message: can't stream video from nbc.com ```
ghostwords commented 6 years ago

Looks like auth.adobe.com subdomains are part of Adobe Primetime's SAML SSO authentication, which may be a distinct problem from OAuth authentication.

Initial solution ideas:

This needs more technical investigation.

ghostwords commented 6 years ago

This happens on SoundCloud, for example:

           fqdn: soundcloud.com
          block: secure.quantserve.com,dpm.demdex.net,5352434.fls.doubleclick.net,5485101.fls.doubleclick.net
    cookieblock: www.facebook.com,apis.google.com,staticxx.facebook.com
       noaction: style.sndcdn.com,a-v2.sndcdn.com,www.gstatic.com,www.googletagmanager.com,ssl.google-analytics.com,connect.facebook.net,vt.myvisualiq.net,tapestry.tapad.com,t.myvisualiq.net,sb.scorecardresearch.com,idsync.rlcdn.com,bcp.crwdcntrl.net,ssl.gstatic.com
        version: 2018.4.23
        message: Login modal dialog buttons don't work.
           fqdn: soundcloud.com
          block: 5352434.fls.doubleclick.net,5485101.fls.doubleclick.net,googleads.g.doubleclick.net,securepubads.g.doubleclick.net,www.google.ca,sb.scorecardresearch.com,ad.doubleclick.net,cm.g.doubleclick.net,ads.yahoo.com
    cookieblock: platform.twitter.com,www.google.com,www.facebook.com,analytics.twitter.com
       noaction: c1.rfihub.net,connect.facebook.net,at.amgdgt.com,static.ads-twitter.com,va.sndcdn.com,i1.sndcdn.com,style.sndcdn.com,a-v2.sndcdn.com,www.gstatic.com,www.googletagmanager.com,ssl.google-analytics.com,rules.quantcount.com,bcp.crwdcntrl.net,loadus.exelator.com,idsync.rlcdn.com,cf-hls-media.sndcdn.com,wis.sndcdn.com
        version: 2018.4.10
        message: When I try to sign in with Google, nothing happens and I don't get logged in.
NeoLegends commented 6 years ago

I'm just writing to letting you know that we had at least one user who wasn't able to login to our web app because of EFF privacy badger. This time it was the combination Spotify OAuth + Spotify-Account-which-was-created-through-facebook.

ghostwords commented 6 years ago

We should look into Ghostery's workarounds:

We currently have multiple different heuristics for allowing third-party cookies in limited cases: ... Redirect-based. When domain a issues a first-party redirect to domain b, we trust b as a third-party to domain a pages for a short time. This handles single sign-on portals which rely on third-party cookies instead of oauth-based methods. OAuth detection. Practical implementations of oauth sometimes require some third-party cookies to be allowed in order to function correctly (Google is the main case). This heuristic detects the OAuth flow in the browser and allows cookies for these cases.

-- https://github.com/cliqz-oss/browser-core/issues/58#issuecomment-394285634

Edit: Here is their onBeforeRequest "pipeline". We are interested in redirectTagger and oauthDetector.

ghostwords commented 4 years ago

Brave seems to allow accounts.google.com by default.

Wiki doc:

When ["Allow Google Login in extensions and third party sites"] is enabled:

  1. It adds a third-party cookie exception for accounts.google.com so sites using Login with Google can work correctly.
  2. It enables chrome.identity for extensions so extensions like Google Keep and Google Calendar can retrieve an OAuth token from google to authenticate users. The OAuth token can be used to retrieve personal information like email id, profile. You can read more about chrome.identity here: https://developer.chrome.com/apps/identity
bkakadiya42 commented 3 years ago

We are also noticing this false blockage issue with google basic auth on our app hosted at abstractops.com. Let me know where we can help to make more progress on this issue? :)

ghostwords commented 3 years ago

In addition, Total Cookie Protection makes a limited exception for cross-site cookies when they are needed for non-tracking purposes, such as those used by popular third-party login providers. Only when Total Cookie Protection detects that you intend to use a provider, will it give that provider permission to use a cross-site cookie specifically for the site you’re currently visiting. Such momentary exceptions allow for strong privacy protection without affecting your browsing experience.

-- https://blog.mozilla.org/security/2021/02/23/total-cookie-protection/