Closed RoxKilly closed 7 years ago
In some cases, my uBO filter lists and settings let something through the cracks and TP actually catches it.
afaik TP uses the same lists (Disconnect) that are already available in uBO. Either your uBO lists are not updated or I'd love to see an example of those cases you're mentioning
@earthlng wrote:
afaik TP uses the same lists (Disconnect) that are already available in uBO. Either your uBO lists are not updated or I'd love to see an example of those cases you're taking about.
You probably have a point there. The Disconnect lists were not selected in my uBO filters. Is Tracking Protection (TP) list limited to Disconnect? Doesn't this link suggest otherwise? The next time I come across such an example, I'll try to remember to post it here; it happens rarely though.
If you guys have made the conscious decision to not cater to people who don't also use uBO then I understand the decision better. If this user.js
is meant to be privacy-enhancing on its own, then I don't understand why TP would be off by default: its upside for privacy is so huge for those without uBO or something similar.
Besides, if uBO Disconnect list is really a drop-in replacement, I don't see the downside of having TP on because that code path will never be traveled anyway for a user with that uBO list active. As my 3rd link shows, TP is only triggered if extensions have allowed the connection to proceed.
@Thorin-Oakenpants wrote:
As for TP, I have read numerous accounts of this being a cause for breakage - so to me personally, I prefer to have one less item to deal with.
Consider: Firefox gives the option of two lists, a basic list (default) meant to have few breaks, and a strict list meant to provide stronger protection. Because the lists are updated every hour (I think) and are maintained by a company that makes it its business to provide privacy tools (Disconnect), I suspect that breakages in the basic list are few and far between. I have no empirical evidence though. Just personal experience (never seen it myself)
@Thorin-Oakenpants I did read the Implementation wiki. I even quote it in the OP. But the only thing it says about uBO is:
There is nothing wrong with running TP [sic] and SB as well as uBlock Origin
This doesn't get across that people who use this without uBO should not expect privacy. If that is your position, consider making it more obvious because as things stand, such a user would be more exposed using your user.js
than using default Firefox in Private Browsing (where TP is ON)
I tried to make the case for Tracking Protection regardless of whether uBO is used; please take another look at my previous post. Thanks for reading.
Is Tracking Protection (TP) list limited to Disconnect?
it seems so:
The blocklist is created by Disconnect according to their definition of tracking.
source: https://feeding.cloud.geek.nz/posts/how-tracking-protection-works-in-firefox/
You can choose between recommended and strict, and that changes urlclassifier.trackingTable
strict: test-track-simple,base-track-digest256,content-track-digest256
default (recommended): test-track-simple,base-track-digest256
If you look at this both base-track-digest256
+ content-track-digest256
are based on this and coming from disconnect.
Thanks for reading.
Pants can sometimes be a bit salty. @RoxKilly, please don't let this spook you away from here. You contribute quality posts and comments, and I'd hate to see you abandoning us.
@earthlng Thanks for the encouragement. I'm in no danger of leaving; I was being sincere with my thanks. I'm curious: what do you think about following assessment:
for people without a uBO-like tool, having TP enabled is a major privacy win. Disabling TP exposes them to tracking
for people with uBO, TP offers little to no added benefit, but no downside either because that code path is never traveled when uBO blocks network requests
so enabling TP is a win for some, and makes no difference to others.
Gets back to my original question: what is the benefit of disabling it?
@RoxKilly wrote:
I'm curious: what do you think about following assessment
I see what you mean but I just can't imagine that someone would use something like this user.js but not use uBO. It can be used simply as a better version of ABP. You don't even have to use the more advanced features and IMO it's really easy to setup and use (in easy mode, at least). Not to mention the amazing Element picker.
so enabling TP is a win for some, and makes no difference to others
it makes the slight difference that it downloads the blocklists (every hour?) basically for no reason at all. #NotSoQuietFox
And as for the sentence you two are arguing about ....
There is nothing wrong with running TP and SB as well as uBlock Origin
I'd say the important part in that is: as well as uBlock Origin
IMHO uBO is an absolute must-have addon, and I'd rather we make that abundantly clear instead of enabling TP (and/or SB)
What is the pref for which TP list to use.
Do you not read, SaltyPants? ;) https://github.com/ghacksuserjs/ghacks-user.js/issues/102#issuecomment-298346502
When changing it in the options FF needs to restart btw
@Thorin-Oakenpants wrote:
What is the pref for which TP [sic] list to use. What setting do we give it?
I don't yet know which preference determines whether the basic or strict protection list is used. I know how to change it in the UI, if that helps: With privacy.trackingprotection.ui.enabled = true
, go to about:preferences#privacy
and click the button labeled "Change Block List" to make the change. My advice is to leave it as the default ("basic list", same as that in the uBO Disconnect filter)
@Thorin-Oakenpants wrote:
We'll also have to make sure the prefs used to get the URLs are reset as well
To get tracking protection working in my browser all I had to do was comment out 0410d
and set 0420
prefs to true
. I did not need to fix the URLs in 0410e
and 0410f
I don't yet know which preference determines whether the basic or strict protection list is used.
seriously? you too? why am I even posting here if nobody reads my shit xD
@earthlng LOL
The download itself doesn't occur every hour. A check is made for whether there is an update every hour:
Every hour, Firefox requests updates from shavar.services.mozilla.com. If new data is available, then the whole list is downloaded again. Otherwise, all it receives in return is an empty 204 response. (source)
@Thorin-Oakenpants I simply commented out those two lines. My copy of user.js
has:
// user_pref("browser.safebrowsing.provider.mozilla.gethashURL", "");
// user_pref("browser.safebrowsing.provider.mozilla.updateURL", "");
In about:config
, these are the values I see:
browser.safebrowsing.provider.mozilla.gethashURL = https://shavar.services.mozilla.com/gethash?client=SAFEBROWSING_ID&appver=%MAJOR_VERSION%&pver=2.2
browser.safebrowsing.provider.mozilla.updateURL = https://shavar.services.mozilla.com/downloads?client=SAFEBROWSING_ID&appver=%MAJOR_VERSION%&pver=2.2
I've also just confirmed that Tracking Protection is working by disabling uBO, then loading nytimes.com
Yes earthlng - I read your stuff.
THANK YOU, ffs :)
Like I said, create a PR
Hell no! I like the way it is right now. Beginning to hate @RoxKilly - now you're 2 against 1 - that sucks! xD
/* 0420: disable Tracking Protection (TP)
* There SHOULD be NO privacy concerns here, but we strongly recommend to use uBlock Origin instead,
* which offers more comprehensive as well as specialized lists. ... ***/
strongly recommend to use uBlock Origin instead
@Thorin-Oakenpants wrote:
Options>Privacy>Using Tracking Protection: "only in private windows" vs "always" - what pref is this again?
"TP only in private windows" is:
privacy.trackingprotection.enabled = false
privacy.trackingprotection.pbmode.enabled = true
Those are the default settings by the way
"TP always" is:
privacy.trackingprotection.enabled = true
privacy.trackingprotection.pbmode.enabled is irrelevant (I just checked)
Blocklist options - if we use strict/simple - this is across all windows right? normal & PB?
I don't know for a fact but I don't see why it would be any other way.
Ok, I'll let you two wrap each other in TP while I go watch some funny cat videos. If you're seriously gonna do this right now, you may want to consider dealing with these too:
pref("privacy.trackingprotection.annotate_channels", false);
pref("privacy.trackingprotection.lower_network_priority", false);
happy TP-ing you two, laterz
Both set as False here, with the URL removed as Roxkilly.
Potential amazon tracking: https://www.robtex.com/dns-lookup/shavar.services.mozilla.com
Browser retrieves the hash for updates from:
https://shavar.services.mozilla.com/gethash?client=SAFEBROWSING_ID&appver=%MAJOR_VERSION%&pver=2.2
EDIT: SAFEBROWSING_ID is not the same as Google API_KEY.
By looking at robtex here, you see the service is hosted at Amazon.
/*** 1200 HSTS
HTTP Strict Transport Security (HSTS) with long duration deployed on this server.
My problem is AmazonCDN hosting and potential tracking because of it's huge online presence.
Huh... missed the whole party here. :) Is there any final verdict? Since the English is not my mother language I have a bit trouble to follow up. ;) What I like, if FF is doing the "protection", is the way FF shows it to the user as seen on http://itisatrap.org pages.... the red page with warning I mean.
To make it work, I have set those:
/ 0410a / user_pref("browser.safebrowsing.malware.enabled", true);
/ 0410a / user_pref("browser.safebrowsing.phishing.enabled", true);
/ 0410d / user_pref("browser.safebrowsing.provider.mozilla.gethashURL", "https://shavar.services.mozilla.com/gethash?client=SAFEBROWSING_ID&appver=%MAJOR_VERSION%&pver=2.2");
/ 0410d / user_pref("browser.safebrowsing.provider.mozilla.updateURL", "https://shavar.services.mozilla.com/downloads?client=SAFEBROWSING_ID&appver=%MAJOR_VERSION%&pver=2.2");
Does my comment make any sense?
EDIT: never think of what @Atavic said... good point EDIT2: I am again in "my folks browsing mode"... they understand red warning page as BIG NO NO... but they don't understand uBo warning... it is simply not red color saying GO AWAY FROM HERE. ;)
Thank you pants. What about SB? I was holding those questions for the lite version, but @RoxKilly opened a pandora box before lite version. ;)
So Mozilla hosts such services on Amazon servers. Will Amazon use the connections to track the browser? They definitely could. as an API key is a unique value that is assigned to a user of the service
I will shut up now. :)
pref("privacy.trackingprotection.annotate_channels", false);
pref("privacy.trackingprotection.lower_network_priority", false);
Note that enabling these won't do anything if privacy.trackingprotection.enabled
is ON. These prefs exist to de-prioritize known trackers when they're not blocked outright. If you enable tracking protection, then the tracking resources don't need to be de-prioritized because they never load at all :)
Browser retrieves the hash for updates from: https://shavar.services.mozilla.com/gethash?client=SAFEBROWSING_ID&appver=%MAJOR_VERSION%&pver=2.2 where SAFEBROWSING_ID is the same as Google API_KEY.
As documented here, SAFEBROWSING_ID
is not the Google API Key. Instead, it gets replaced with the contents of browser.safebrowsing.id
(which defaults to navclient-auto-ffox
).
@Atavic , the use of HSTS by itself does not mean fingerprinting. HSTS is actually a great security feature because it protects against a man-in-the-middle downgrade attacks by forcing the browser to connect only over a secure channel by default.
Using HSTS to fingerprint requires having the browser make many connections to many domains (subdomains usually) and testing whether the browser knows to connect to these domains over HTTPS by default. So unless we have evidence that the Tracking Protection server connection involves many domains with HSTS there can be no fingerprinting of that sort.
In addition, Firefox doesn't save HSTS settings in Private Browsing mode, so in the browser's default configuration (Tracking Protection used during Private Browsing) this fingerprinting technique wouldn't even work. Again, I think we need evidence that there is even fingerprinting going on in the first place for this to be a concern.
@fmarier
I can confirm that my browser.safebrowsing.id
is simply set to navclient-auto-ffox
, so there is no inherent fingerprinting either in the URL that connects to the blocklist server:
browser.safebrowsing.provider.mozilla.gethashURL
=> https://shavar.services.mozilla.com/gethash?client=SAFEBROWSING_ID&appver=%MAJOR_VERSION%&pver=2.2
browser.safebrowsing.provider.mozilla.updateURL
=> https://shavar.services.mozilla.com/downloads?client=SAFEBROWSING_ID&appver=%MAJOR_VERSION%&pver=2.2
I'm having a hard time seeing how there is tracking going on. Seems to me that at most, the Amazon hosted server shavar.services.mozilla.com
will know that a Firefox browser at my IP is using the Tracking Protection blocklist. This is true in default Firefox installs though so it's useless as a fingerprint vector.
I keep reading that uBo has the same default lists as TP.
Which lists are people talking about? When I compare the Disconnect tracking list in uBo to the TP lists, they appear very different.
When I compare the Disconnect tracking list in uBo to the TP lists, they appear very different.
Indeed. No idea why that is. But TP thinks allowing google-analytics.com (among others) on google is totally fine. I can't take TP serious when it does things like that. The mozilla version moved the questionable domains from "properties" to "resources" (compared to the original) but idk how much of a difference that makes.
edit: just tested in a vanilla FF with TP enabled and googlesyndication.com was not blocked, and that's in the same list and category as google-analytics.com. "recommended" or "strict" makes no difference for this. I'm sorry but TP is a joke. I guess we could disable the whitelist part but why bother - use uBO and be done with it
@Roxkilly Thanks for reminding me that Private Browsing mode allows to surpass the potential privacy problem.
Regarding AmazonCDN servers, I keep an eye on my browser's connections with TCPView by Sysinternals and these servers are almost always present on top sites.
Is there a potential tracking issue here? Yes! (that's enough to me)
It's also worth noting that if a browser got a single HTTP -> HTTPS redirect and then picked up HSTS on one site, it'd never see it again on any other site using the CDN.
Source: https://github.com/MaxCDN/bootstrap-cdn/issues/750#issuecomment-240039399
Also, an interesting read as Example of HSTS cross-origin history sniffing (Page 28): http://web.mit.edu/zyan/Public/appseccali.pdf
Is browser.safebrowsing.provider.mozilla.gethashURL
even used for TP? I don't think so
Indeed. No idea why that is. But TP thinks allowing google-analytics.com (among others) on google is totally fine. I can't take TP serious when it does things like that.
TP is designed to block third-party tracking. If you're visiting GMail for example, they are clearly able to track you regardless of whether or not you block google-analytics.com
. They have you in their server logs and they can run all the JavaScript they want in your browser.
On the other hand, Google shouldn't have to know that you're visiting CNN so google-analytics.com
will be blocked by TP there.
Is
browser.safebrowsing.provider.mozilla.gethashURL
even used for TP? I don't think so.
Not currently. The TP list doesn't need it since it contains the full hashes, not the partial hashes like the other Safe Browsing lists.
@earthlng wrote:
But TP thinks allowing google-analytics.com (among others) on google is totally fine. I can't take TP serious when it does things like that...and googlesyndication.com was not blocked
Technically, google-analytics on a google site is not 3rd party tracking. Also consider: The objective of TP is to reduce tracking without breaking sites. Where sites break, functionality may supersede privacy. Firefox (or Disconnect) may have discovered that in the wild, a lot of sites were breaking. Mozilla even wrote a special article about the breakage here.
What uBO does -- and one reason it's such an amazing tool -- is use its redirect
syntax to replace the tracking functions with a neutered one, so that sites won't break. For people without uBO & Co., Tracking Protection is a big deal and it significantly reduces 3rd party exposure; I wouldn't be so dismissive.
@fmarier, thanks. But the google-analytics.com scripts are doing all (or most) of the tracking, don't they?
So how does that work exactly - ELI5 please - all the domains in "properties" can load whatever they want from the domains listed in "resources" ? (list here) I'm confused because you mentioned gmail but that is listed under "resources". Or can all domains in "properties" AND "resources" access all domains in both lists? Just out of curiosity.
Is
browser.safebrowsing.provider.mozilla.gethashURL
even used for TP? I don't think so.Not currently.
Why would it change and suddenly use it too, if it works perfectly fine without it?
@RoxKilly so you're trying to tell me that google.com would break if google-analytics.com would be blocked? Come on - even if that was the case it was deliberately created that way, exactly because, as I suspect, google-analytics.com scripts are google's main tracking feature. I can use google.com with scripts blocked and uBO doing its thing in the background - no problems at all.
I wouldn't be so dismissive
TP is definitely better than no protection but I'm arguing in the context of this user.js here
But the google-analytics.com scripts are doing all (or most) of the tracking, don't they?
That's a question for Google :) They certainly have the technical capability to track their users in lots of other ways when they connect to their own servers.
I would assume that they have the ability to track me when I use their services regardless of the add-ons / privacy settings I have. The possible exception would be if I'm using the Tor Browser Bundle.
all the domains in "properties" can load whatever they want from the domains listed in "resources" ?
Yes, that's essentially how it works.
I'm confused because you mentioned gmail but that is listed under "resources". Or can all domains in "properties" AND "resources" access all domains in both lists?
GMail is confusing because the app is on google.com
(which is in properties
) and not gmail.com
.
so you're trying to tell me that google.com would break if google-analytics.com would be blocked?
For an example where things are quite broken, try blocking twimg.com
all the time. Twitter is unusable without it. With the entity list, we can group twitter.com
and twimg.com
together (since they are the same company) and eliminate that breakage while blocking twimg.com
as a third-party tracker on other sites.
That's a question for Google :)
I doubt that they would be all too willing to let us know :( "Don't be evil" my ass
Yes, that's essentially how it works.
Essentially? with exceptions? if ELI5 is too much to ask, ELI12 or so is fine too :) nvm, open-source
twimg.com
allowing domains that host primarily css/images vs allowing domains who's sole purpose is to serve "analytics" scripts is quite different.
edit: In retrospect my statement about TP being a joke was admittedly a bit harsh. Maybe you were hinting at this (without feeling at liberty to say so directly maybe), but I see now that if you blocked GA & co on google etc, they could just adapt to it and work around it - not to mention maybe being pissed off a bit and potentially throwing you a smaller bone next time ;)
It's great that TP is included in FF, even enabled by default in PB, and I'm sorry if I offended someone. Keep up the great work
(and NO - Putin did not make me edit that [#CNN] - hindsight is a bitch)
@earthlng What Disconnect list are you using in uBo to cover a superset of everything in TP?
in uBO? none. I block domain lists in uMatrix. Apart from the default ones I use these (with limited usefulness)
https://s3.amazonaws.com/lists.disconnect.me/simple_ad.txt : 0 used out of 2'703
https://s3.amazonaws.com/lists.disconnect.me/simple_malvertising.txt : 568 used out of 5'337
https://s3.amazonaws.com/lists.disconnect.me/simple_malware.txt : 0 used out of 2'601
https://s3.amazonaws.com/lists.disconnect.me/simple_tracking.txt : 0 used out of 34
Disconnect doesn't exactly make it very easy to find these, so if you know of better ones that aren't included by default in uBO or uMatrix, please share
@earthlng wrote:
so you're trying to tell me that google.com would break if google-analytics.com would be blocked?
No I was referring to other sites that rely on google-analytics in a way that breaks the site if the analytics aren't loaded properly. Anyway I feel like we're going in circles. We're in general agreement (uBO > TP > nothing). I only jumped in because you initially hinted that TP was a joke. but I took exception to that.
In #103, the great @Thorin-Oakenpants wrote:
TP uses the same list as uBo (note: there are two lists, simple+strict, default is simple and we would leave it at that but include the pref for info/enforcing strict)
What are you talking about? What list? Is uBo configured by default to use this list? Are these lists always identical?
@Thorin-Oakenpants No problem. Although it's certainly likely, I haven't seen anything conclusive that uBo (especially out of the box) actually covers everything that TP covers. Of course, uBo (or any content blocker) can block whatever TP blocks, but that requires a list in a compatible format.
uBO doesn't use the Disconnect list by default. Here is its default setup. The 3rd party lists loaded are:
Yes. I know this because as I pointed out in the OP:
In some cases, my uBO filter lists and settings let something through the cracks and TP actually catches it. This is usually a tracking image of some sort
I don't use the Disconnect list in uBO, though I have EasyList, Peter Lowe's, and EasyPrivacy ON. These occurrences are rare though; the next time I run across one, if I remember, I'll post it here
Those I have additionaly to uBo standard for TP:
https://raw.githubusercontent.com/quidsup/notrack/master/trackers.txt
https://raw.githubusercontent.com/piperun/iploggerfilter/master/filterlist
https://raw.githubusercontent.com/metaphoricgiraffe/tracking-filters/master/trackingfilters.txt
https://filters.adtidy.org/extension/chromium/filters/3.txt
I don't go FB, but to whome who does:
https://easylist-downloads.adblockplus.org/message_seen_remover_for_facebook.txt
I have also other collections for SB and others, if anyone interested. Maybe this (uBo usage sharing information) should go into separate topic?
EDIT: If anyone sees some in this list that are total nonsense, please let me know ;)
@Thorin-Oakenpants wrote
thanks roxkilly : check the pretty picture above. My question now would be, is there ANYTHING covered in TP that is not covered by uBo's default?
I just checked these and updated: Basic tracking list by Disconnect - 29 used out of 34 Malvertising filter list by Disconnect - 639 used out of 5,334
Thanks. On the surface, those results indicate that much is covered by Disconnect (and hence possibly TP) that is not covered by the default uBo setup.
I'm not sure if the analysis is so simple, however. Wouldn't it depend on which order uBo processes lists? IOW, if 2 lists have foobar.com
, then I think uBo would count the item as "used" in one list and not the other.
What I really miss in uBo is statistics... but I understand why those can't really be implemented. If those would be implemented the uBo would take a lot of CPU resources to do so. What I mean by stats is that uBo would have a counter per list, how many hits overall and how many are unique to the specific list. With that info we could easily decide, which lists are really needed and which are just sitting there doing nothing.
Crap, this comment might be just a spam here and should go to uBo "issue" instead.
Under Malware filter list by Disconnect I have 2 used out of 2598, checked now. Two days ago I had 0 used out of XXX. But I don't use this lists for obvious reason as they are covered by some other lists I am using.
I think what @crssi means is a counter about which filters (and from which list) you actually encounter in your browsing.
Doh... my English again. :)
Yes, @earthlng, you are correct.
I was thinking about this issue a bit more, and a strong argument to keep TP enabled is that its list can be automatically updated multiple times per day (I believe).
To the contrary, most uBo lists are only automatically updated every 3 days, some only get automatically updated every 7 days, and I believe some may be scheduled to take even longer.
Yes, you can manually force uBo updates more frequently, but it's not really reasonable to expect users to all do that.
@earthlng I tend to agree. And at the same time, I tend to disagree. :)
I agree because when the frequency of updates is infrequent, then looking for updates frequently isn't productive.
I disagree because it doesn't take any effort from the user, and only the most severely bandwidth-restricted users will notice. Also, sometimes a really invasive tracker can come along and needs to be blocked ASAP. That, as you pointed out, is relatively rare.
Now, when it comes to SB (vs TP), I am starting to think it's really important to have blocklists updated more frequently than every 3 days. And SB provides that. From what I can tell, SB would have blocked the phishing scam described in https://www.ghacks.net/2017/05/04/fell-prey-to-google-docs-phishing-scam-do-this/ but uBo would not have blocked it for a majority of its users.
The implementation guide explains:
Could someone please elaborate? Is the block because the browser has to connect to a remote server to download blocklists? @Thorin-Oakenpants do you operate under the assumption that this
user.js
must be used in conjunction with uBlock Origin? If this is meant for both uBO users and non-users, why disable TP?Setting that aside, let me make the case for TP even for a uBO user like myself: For the vast majority of webpages, TP never plays a role because uBO blocks requests before they get to the TP code (see the last comment from link 3 below). So there is no additional burden on the browser and I don't see an additional privacy exposure (beyond the blocklist downloads).
In some cases, default uBO filter lists and settings let something through the cracks and TP actually catches it (eg: enable Tracking Protection and open this page as of May 4 2017). This is usually a tracking image of some sort. In those cases I'm glad to have TP on.
For Reference