arkenfox / user.js

Firefox privacy, security and anti-tracking: a comprehensive user.js template for configuration and hardening
MIT License
9.92k stars 512 forks source link

4.1 Extensions suggestions [CookieBro + Chameleon + PrivacyBadger] #776

Closed rugabunda closed 4 years ago

rugabunda commented 5 years ago

https://github.com/ghacksuserjs/ghacks-user.js/wiki/4.1-Extensions

Cookiebro is by far the best cookie cleaning addon to date; https://addons.mozilla.org/en-US/firefox/addon/cookiebro/

  1. The cookie log gives a realtime log of cookies as they appear... this and the interface allows a user to blacklist individual cookies on individual sites, or globally with wildcards, whereas cookie autodelete only allows to delete all cookies on a given site. It has its own cookie editor as well.

  2. It effectively deletes cookies left behind by cookie autodelete, which can leave hundreds of cookies in place even after clicking to clean or using the whitelist approach. Using both cookie autodelete and "self destroying cookies", they left behind cookies from ad and marketing sites, some of them dating back 4 months or more, even after explicitly using the "clear" function. Cookiebro deletes them all. Its also 32% smaller than cookiebro.

  3. Privacy warning: fetches favicons from google for the cookie log viewer to better discern the websites listed. These connections are visible in browser console, or about:cache?storage=disk&context= Disable this setting to increase privacy. Google can also drop google NID cookie, which is both a settings & targeted tracking cookie, when pulling favicons from google.com, though this cookie was not always dropped in other firefox browsers in testing this extension.

Browser Consol

  1. Use a whitelist approach, whitelist only those sites you login to, and set to delete unwanted cookies on a timed interval to not break sites, and ensure no tracking cookies, zeroday or otherwise, sneak by all other protections long enough to track you. It has an extremely intuitive interface for just whitelisting and blacklisting.

  2. Best set with mozillas built in third party tracking blocking, to prevent site breakage. Blocking third party cookies in total can break sites.

Enable:

Auto-delete unwanted cookies at browser startup Auto-delete indexedDB and localStorage of websites at browser startup Auto-delete pluginData (e.g. Flash Cookies) of normal websites at browser startup Enable blacklist filtering Delete ETag HTTP Response header from blacklisted sites with no matching whitelist setting Delete Link HTTP Response header from blacklisted sites with no matching whitelist setting Delete unwanted cookies every 30 min

Blacklist example:

*.*/*GeoIP* [globally block any cookie that contains the name GeoIP] github.com/has_recent_activity [on github block recent_activity cookie] *.*/__cfduid [block cloudflare tracking globally]

Chameleon, I agree with your statement, while it is great for experimenting, its wise to use a minimalist approach... It can enhance security greatly by enabling two options only, that is

  1. the most basic static user agent switch, ex. to Safari on Mac (malicious sites and state actors will drop malware and use targeted exploits for Mac/Safari on my windows machine, or visa versa. This is a huge security improvement on its own) I found only skype.com [for login] & presstv.com [for video player] needed whitelisting for this setting after a year of browsing, and

  2. "script injection", which is very important, and necessary for spoofing not to be detected by Java scripts. I was using "User-Agent Switcher and Manager" previously, the spoofing was detected on https://whoer.net/ . Spoofing succeeded in the header, but failed in java scripts. Using Chameleon with Script injection fixes this leak.

Privacy Badger

Contrary to what is stated on your wiki page PrivacyBadger does do things ublock origin wont do, for example, because it learns as it goes without whitelists or blacklists, it will detect unknown trackers that have yet to be flagged by the ublock community... and it deals with referrers, third party tracking cookies and supercookies in local storage, first to third party cookie sharing via image pixels, canvas fingerprinting, automatically prevents social media widgets, domains and cookies from tracking a user across the web in a very convenient way that does not hide the element... which makes whitelisting much simpler than Ublock. This means it is redundant to use social media blocklists in Ublock Origin while using privacy badger.

PrivacyBadger uses algorithmic learning and not a blacklist, so anything that Ublock has already blocked will appear in the green in privacy badger, even if it is a tracking company. It blocks a LOT of cookies and tracking websites with Ublock origin already installed with all blocklists enabled. Your list of automatically detected, blocked and restricted domains in PrivacyBadger will become quite large. In my case, 1846, 1152, 972 respectively from most used, to least used browsers. PrivacyBadger on top of UBO has great benefits, with no negative impact on site functionality.

kah0922 commented 5 years ago

Cookiebro isn't open source though.

Thorin-Oakenpants commented 5 years ago

What happened to using the specified issue?

:small_orange_diamond: Relevant Links

But since it's here, and rather bulky.... with three items ...

Thorin-Oakenpants commented 5 years ago

CookieBro: disclosure: I have not used or installed or tried the extension

fetches favicons from google for the cookie log viewer - w&^*#!! : https://addons.cdn.mozilla.net/user-media/previews/full/199/199415.png - there's no need for this IMO, there are other ways to assign a visual. OK, so you can disable that in the settings, right?: but it should be opt-in. Assuming it's opt-out, this inspires zero confidence in me about someone who is supposed to be helping your privacy. Maybe I'm missing something here

I'm also not inspired by the single email contact as a go-to.

It's got a lot of bells and whistles, extras. When extensions start to bloat with feature creep it worries me. Maybe I'm biased here. I'd rather specialized extensions that focus on one thing, and do it well. I'm also not that worried about being able to inspect cookies - that's what dev tools are for.

Question: can cookie bro delete all persistent storage in-session. cookies (yes, obviously) - but what about IDB? Why is this any better than the shortcoming of any cookie extension regarding clearing IndexedDB, Service Workers cache, appCache, or cache by host.

I actually don't want to recommend any cookie extensions, because they give a false sense of privacy

Thorin-Oakenpants commented 5 years ago

I don't see a case need for Chameleon.

point 1: says you are Firefox spoofing as Safari - now you're pretty much unique: both passively and actively. This information paradox makes you stand out for a perceived improvement in security. Browsers are already secure (nothing is ever perfect) and always being worked on in this regard. 99% of this can be mitigated by the end-user not being stupid (and there is no magic solution for fixing stupid). It is also trivially easy to detect your browser and version and OS.

for example

point 2: "Spoofing succeeded in the header, but failed in java scripts" - that's because the extension you were using wasn't good enough. Script injection is not unique to Chameleon.

We do not recommend spoofing outside of RFP. I get that there may be edge cases, in which case use a secondary profile or portable Firefox or whatever: hell, use Chrome for google services (I do). Why compromise/relax your main browser for some sites? Or if you really need to, in the case of User Agents or time zone spoofing, then use an extension, as long as it has a blacklist all, and only applies to a whitelist: that is RFP is the default, and the extension is the exception (PS: not sure if timezone spoofing via an extension overrides the RFP code: it depends on how it's coded: e.g canvas can't be overriden by CanvasBlocker, until you set a site exception: but not every RFP has site exceptions)

So, long story short: Chameleon = NO

Thorin-Oakenpants commented 5 years ago

Privacy Badger is probably worth a look. It doesn't really add anything (without looking further) than a hardened or medium(?) uBO setup. Most uBO users would probably set and forget (just bock ads i.e they only really use the filter lists) - and in this regard, badger might be worth recommending.

Does it use CSP header modification? If so then that's a problem.

PS: why aren't badger users feeding the badger data to uBO filter list maintainers @gwarser - maybe they are and you list is never pruned? IDK

gwarser commented 5 years ago

Does it use CSP header modification?

Does not look like it does.

PS: why aren't badger users feeding the badger data to uBO filter list maintainers

No way to export this data?

It doesn't really add anything (without looking further) than a hardened or medium(?) uBO setup.

Can block spills not catched by filter lists. In uBO medium mode can still catch tracking pixels (for ex. external images are not globally blocked). In hard mode it may help on pages where you allow some content, but I don't think it will have chance to learn.

Thorin-Oakenpants commented 5 years ago

No way to export this data?

IDK, was just posing the question. NFI what the data even looks like, but it sounds like a great source for the filter list maintainers. Someone should go ask them :)

rugabunda commented 5 years ago

"CookieBro: disclosure: I have not used or installed or tried the extension" clearly. I stand by my suggestion that it is the best addon there is for cookies, hands down. It is by no means bloated, and at 234.5 KB, is 32% smaller than cookie autodelete, it has everything one needs in one very convenient package, and it actually does what it says it does, whereas cookie autodelete doesn't. Its basically a fine grained cookie firewall. I recommend people read my comment above again, everything you need to know is in there more or less. I have suggested to the developer to disable favicons by default and add a privacy warning. He also uses optional favicons in FeedBro (RSS feeder addon). I imagine that is where he got the inspiration... FeedBro also includes an option to pull icons from duckduckgo, OR google. I wouldn't have a problem with duckduckgo.

The only thing I could ask for would be deleting cookies on tab close, though this is probably by and large redundant with a short timed removal; Given its benefits and the fact it actually deletes cookies as it says it will, periodic removal will effectively prevent ad companies from creating a profile on a user.

Sorry I did not realize there was a section for suggestions in the link section.

rugabunda commented 5 years ago

Update from dev "Thank you for the feedback!

We'll disable the favicons by default in the next version!

Dev"

rugabunda commented 5 years ago

I sent the email asking for the change 21 minutes ago. I got this email: Cookiebro 2.10.1 is now available for Firefox and it has favicons disabled by default. We also published a new version to the Chrome Web Store but it's pending review for a while.

Thanks again for the heads-up!

Dev

nodeticswww commented 5 years ago

Thanks a lot for @rugabunda for the praise and recommendation! We really want to provide the best cookie manager possible.

Some clarifications:

  1. When blacklisting cookies by name, domain name matching supports wildcards but cookie name has to be an exact match. So contrary to the example, "*.*/*GeoIP*" doesn't work. It would have to be "*.*/GeoIPtracker" (or whatever the cookie name is). Naturally you can also blacklist entire domains without specifying a particular cookie name.

  2. Cookiebro source code isn't obfuscated or minified in any way so you can review the code in the browser. It's 100 % clean and safe.

  3. Cookiebro can automatically or manually remove IndexedDB, localStorage and pluginData (see Options).

Cookiebro Dev

rugabunda commented 5 years ago

@nodeticswww thank you so much for clarifications and incredibly speedy response. I appreciate your upcoming modifications. I am excited to hear the next release will include an option to use duckduckgo for favicons. I know disabling favicons is potentially somewhat of a sacrifice on first impressions of the GUI... but I believe it is worth disabling it given the potential implications. Thanks for caring about your users privacy.

rugabunda commented 5 years ago

I don't see a case need for Chameleon.

point 1: says you are Firefox spoofing as Safari - now you're pretty much unique: both passively and actively. This information paradox makes you stand out for a perceived improvement in security. Browsers are already secure (nothing is ever perfect) and always being worked on in this regard. 99% of this can be mitigated by the end-user not being stupid (and there is no magic solution for fixing stupid). It is also trivially easy to detect your browser and version and OS.

I have on numerous occasions visited questionable websites that had dropped malicious dmg files instead of exe files onto my windows machine. This was a clear message to me that the automated systems for malware distribution were effectively mitigated by this simple modification; even if a website had the capability of detecting spoof on some level, they wouldn't know the exact browser or OS version, greatly reducing the chance of any form of any successful targeted attacks. All of the fingerprinting sites I used believed I was Mac/Safari. I also have modified dozens of TCPIP stack & IPV4 settings, randomized identifiers, disabled TCP fast open (cookies) & time stamps, disabled Tcp1323Opts, PMTUDiscovery, deadgwdetect, icmpredirect, ipsourcerouting, multicastfowarding, pmtubhdetect, IGMP, auto DHCP and all the rest. So I doubt my O/S can be fingerprinted as any particular O/S at all, not even on the local network. Nmap was not able to detect an OS, only that a machine existed.

I also have enabled resist fingerprinting in firefox so I imagine that will help quite a bit also. Thanks for the heads up on RFP. I realize Safari/Mac sits somewhere around 12% of the internet traffic population. I believe sacrificing a bit of uniqueness for the security that has been demonstrated is currently worth it... I've seen its benefits.

We do not recommend spoofing outside of RFP. I get that there may be edge cases, in which case use a secondary profile or portable Firefox or whatever: hell, use Chrome for google services (I do). Why compromise/relax your main browser for some sites? Or if you really need to, in the case of User Agents or time zone spoofing, then use an extension, as long as it has a blacklist all, and only applies to a whitelist: that is RFP is the default, and the extension is the exception (PS: not sure if timezone spoofing via an extension overrides the RFP code: it depends on how it's coded: e.g canvas can't be overriden by CanvasBlocker, until you set a site exception: but not every RFP has site exceptions)

So, long story short: Chameleon = NO

Yes the whitelist approach makes sense and is offered by chameleon, user agent can be applied uniquely and individually on per site basis in the whitelist, then randomized everywhere else or set static RFP or whatever. Rather than using another browser you can take the primary offending hegemons like google, youtube, twitter, amazon, facebook, and give each of them static unique user agent strings, and keep the rest of the web RFP or as I prefer, RFP + another O/S for security purposes.

I also love the fact chameleon has a checklist for all the most important privacy elements in about:config... its a tight little addon.

rugabunda commented 5 years ago

Privacy Badger is probably worth a look. It doesn't really add anything (without looking further) than a hardened or medium(?) uBO setup. Most uBO users would probably set and forget (just bock ads i.e they only really use the filter lists) - and in this regard, badger might be worth recommending.

Does it use CSP header modification? If so then that's a problem.

PS: why aren't badger users feeding the badger data to uBO filter list maintainers @gwarser - maybe they are and you list is never pruned? IDK

@Thorin-Oakenpants yeah most people would find medium and hard UBO extreme, the badger automates everything you would have to do manually. It only blocks what ublock does not. So you can test it out and see if it finds anything that slips through.

Thorin-Oakenpants commented 5 years ago

All of the fingerprinting sites I used believed I was Mac/Safari

That's because they're not looking very hard. I bet you can't hide it from me, and I'm not some wizard with special skills - so you can be damn sure that others already know how, and that it is used in the wild.

I don't want to regurgitate my knowledge and write another book here, so I'll try and keep this simple.

FP'ing test sites (such as Panopticlick, amiunique, etc) are only good for showing the metrics they can pull out. But they are often very limited, and do not keep up with the latest methods. They are useless for entropy figures because the data is tainted by the nature of the visitors, and repeat visits of said users as they tinker -> leading to self-fulfilling prophecies (users always try to defeat the highest entropy items first) that drive those down to absurd levels. They also do not really factor into account buckets of users, such as Firefox users (you cannot hide your browser, trust me: or your OS). And the sets can contain long-lived data that is out of date and skews the results. When it comes to FP'ing entropy, you have to trust in the math and science behind the methods used to thwart it: and take other factors into account such as enforcement of metrics in a set of users: such as Tor Browser users.

even if a website had the capability of detecting spoof on some level, they wouldn't know the exact browser or OS version

A lot of spoofs can easily be detected, including randomization. Of course they know your browser (if they want to, including the version), and of course they know your OS (if they want to: i.e major platform: e.g Windows, Mac, Android, Linux: and on Linux they can often detect the distro). Just because they probably don't check that, doesn't make your solution any good. Security by obscurity is not real security. Security is a multi-layered thing: and if you can't handle browsing the web, including dodgy sites, without trying to lie about your OS, then you're not doing it right.

If you understand how entropy works, then spoofing your browser and fiddling with your networking FP is only making you super unique. Especially passive FPing such as your networking. The point is that I said TCP/IP stacks was an example of how your OS could be discovered: What about the other two examples?: and I have many more. You cannot hide your OS. And you cannot hide your browser or version.


Chameleon

I won't be adding it as a recommended extension. As I already said

I get that there may be edge cases, in which case use [option a] [option b] or if you really need to [snip] then use an extension

I'm not going to promote extensions that conflict with RFP and raise entropy. It's diamterically the opposite of what Tor Browser and RFP are aiming for. And I don't want to overload the wiki page with numerous extensions. If the end-user needs to get around some limitation or breakage of RFP, then they already have multiple choices. In fact this is what RFP is for in the future: to be able to apply it to a Super Private Browsing Window - i.e users browse anonymously via tor in this window, but for breakage, they can fall back to a PB mode, or a normal window.

I also love the fact chameleon has a checklist for all the most important privacy elements in about:config

Yes, I have spoken with sereneblue and tested (and pointed out fixes needed for) his anti-FPing in the past and I check in on his repo weekly to see what's happening. I respect him highly: I can see it in his knowledge, and how fast he responds and keeps his repo clean of issues


CookieBro

It is by no means bloated

I didn't say it was bloated: I said "feature creep". Size has nothing to do with it. I was a bit quick and harsh with that wording (I had a cursory glance at the features on AMO and saw a long list including etags:and duplication of features already available in FF, but to be fair, they're just a logical expansion of dealing with cookies). I also didn't say the code was obscured or minified, or not open source.

I'm actually, as already said, against cookie extensions in general: as they create a false sense of privacy. I haven't used, or needed one myself for about 4 years: I have a better way (i.e I block all cookies by default: cookie permissions control localStorage, IDB and more ), and we don't support Flash (so I don't care about LSOs), and appCache is practically deprecated on the web, go look at Mozilla's telemetry (and we disable it), and so on. I actually never have to clean anything.

Furthermore: FPI (first party isolation) mitigates a lot of damage. Additionally, there is a superior option in dealing with persistent local web data: and that is Temporary Containers. And I'd rather push that message.

I would still like to know, as already asked, in what way is CookieBro any better than say CookieAutoDelete when it comes to clearing persistent local data in-session - such as IDB, SW cache? I don't care if it clears stuff on startup, or on close. And I don't care if you can view and edit individual cookies

Give me a list of things CBro does that CAD doesn't,

rugabunda commented 5 years ago

If you understand how entropy works, then spoofing your browser and fiddling with your networking FP is only making you super unique. Especially passive FPing such as your networking. The point is that I said TCP/IP stacks was an example of how your OS could be discovered: What about the other two examples?: and I have many more. You cannot hide your OS. And you cannot hide your browser or version.

Well said, point taken... yes I suspected as such, and the more I look into this the more I understand, I am new to canvas printing and fingerprinting, thanks for the education!

Chameleon

I won't be adding it as a recommended extension. As I already said

I get that there may be edge cases, in which case use [option a] [option b] or if you really need to [snip] then use an extension

Makes sense, I will be reverting to RFP alone and see how that treats me security wise over a few months.

CookieBro

It is by no means bloated

I'm actually, as already said, against cookie extensions in general: as they create a false sense of privacy.

could you explain?

I have a better way (i.e I block all cookies by default: cookie permissions control localStorage, IDB and more ),

do you use a global umatrix rule or firefox whitelist control?

Furthermore: FPI (first party isolation) mitigates a lot of damage. Additionally, there is a superior option in dealing with persistent local web data: and that is Temporary Containers. And I'd rather push that message.

I would still like to know, as already asked, in what way is CookieBro any better than say CookieAutoDelete when it comes to clearing persistent local data in-session - such as IDB, SW cache? I don't care if it clears stuff on startup, or on close. And I don't care if you can view and edit individual cookies

Give me a list of things CBro does that CAD doesn't,

I would recommend you test it out and see for yourself. For one, cookiebro actually deletes all the cookies not in a whitelist, whereas hundreds of cookies, many months old, still remained in the browser while using cookie auto-delete and self-destroying cookies. My guess is this happens to anyone who does not block cookies globally and those whom allow third party cookies. Many reviewers on the addons page said the same thing. This left me curious if there was not some agenda at hand in these other addons. Given its open source then I find it highly unlikely.

the author claims

Even though third party cookies are cleared with this extension, it is better to disable third party cookies from the settings

This is obvious, I block third party tracking only to prevent site breakage... cookiebro is able to delete all third party cookies without any problem at all. CAD does not seem to be able to remove a lot of third party cookies.

The secondary thing cookiebro offers is an intuitive realtime cookie firewall/log that allows you to view and whitelist or block individual cookies or domains globally... including those dropped by extensions web connections. There a user can learn about each cookie dropped individually, by simply searching for them. I never really had as much intimate knowledge of cookies before using this addon, their names were too obscure and meaningless for me to bother, or you had to load a cookie editor to view them, or some other browser element, but you this addon really simplifies and allow users to educate themselves to see exactly what is going on behind the scenes.

CAD claims they delete localstorage one time.

[clear local storage] Upon enabling the setting, since the browsingData API doesn't allow for enumeration, all localstorage will be cleared upon enabling the setting. This only happens once so you start with a clean localstorage.

They do not explicitly state they delete storage on tab close, and people are having a lot of troubles with it. I had dozens of local storage left in the browser, and hundreds of third and first party cookies left behind. https://github.com/Cookie-AutoDelete/Cookie-AutoDelete/issues/556 [2]

cookiebro deletes localstorage, indexdb & plugindata on browser start up

I would love if they had the ability to delete on tab close as well, including service workers. I will ask the author about this, he is very responsive.

rugabunda commented 5 years ago

From the author "Cookiebro 2.10.2 now has favicons enabled by default again but the favicon provider is DuckDuckGo which doesn't set any cookies or track you. So now users can safely keep favicons on as well without worrying about Google related cookies. "

Thorin-Oakenpants commented 5 years ago

do you use a global umatrix rule or firefox whitelist control?

I use Firefox's internal settings to block all cookies and then site exceptions to either allow cookies permanently or for session only. I have about ten exceptions in total: half are session only so the site will work, the others are permanent for logins. All other sites I visit seem to work without cookies. People will get different mileage depending on how they use the internet. For me this works a treat.

FF's internal mechanism needs to be used, as this directly controls access to localStorage, IDB etc. Otherwise, FF will allow sites to use said persistent web storage and then the user has to deal with controlling it (and some APIs are lacking).

uMatrix does not stop cookies being set: it only stops them being sent back in headers: javascript cookies are not blocked at all

could you explain?

Because they cannot clear IDB and other persistent web storage by host

No one cares if you can clear data on start-up or close. Firefox can do that for you anyway. It only matters that you can clear in-session. Extensions that wipe cookies in session but leave orphaned data, are not (fully) effective. They are snake oil at worst, IMO (sorry for the strong wording), and a false sense of privacy at best.

I'm only concerned about the sanitizing here. Extras such as blocking cookies based on rules, or being able to view and edit individual cookies is all fine. Blocking, whitelisting is all fine.

rugabunda commented 5 years ago

Is it not possible for addons to simply take advantage of firefox inbuilt features, and act as a front end to help automate and streamline this whitelist / blacklist process, like the WFC firewall, and clear in-session storage?

I linked the CAD issues in my previous post. ( Cookie-AutoDelete/Cookie-AutoDelete#556 [2] )

rugabunda commented 5 years ago

@Thorin-Oakenpants, so I am testing firefox inbuilt site data / cookie management permissions, and I see zero benefits here. I created a whitelist of only those sites I want to save cookies / data, immediately went to youtube.com, and it dropped cookies and site data. In other words, a user must literally manually blacklist every single non whitelisted domain and subdomain individually using firefoxes buried option menu just to block cookies and localstorage. I have to do it twice if I am using a sandbox. There is no export. Firefox combines deleting both cookies AND site data, and only upon browser close. Although I have whitelisted I am afraid to enable this, I do not want to be logged out of websites. I don't see how you are blocking localstorage; are you using hidden about:config preferences? or a painstaking time consuming blacklist? I see no benefit here at all. Cookiebro is already doing the job perfectly, and there is no garbage ever left behind that I can see on browser restart.

Ok I see one service worker in firefox, and will test if this service worker is cleared on browser restart by cookiebro or automatically by firefox.

rugabunda commented 5 years ago

i saw two service workers in about:serviceworkers immediately upon loading the browser, it was the last page open, then refreshed the page and they were gone milliseconds later. this is the behavior of cookiebro I believe, as it is set to clear data on browser startup. I will do some further testing.

rugabunda commented 5 years ago

I was incorrect, cookiebro does not currently clear SW, firefox does however, and firefox does this automatically upon browser start. Bear in mind if there was a SW and you were on VPN, even an extension that clears on tab close means you have to close the tab or the browser if you have multiple tabs. Sounds wiser just to disable session workers altogether if you are using a vpn. Better in a dedicated virtual machine. Or if a extension deleted SW and all local storage/cookies when https/http connections fail / reconnect, but given this may break sites, using it in a dedicated extension for vpns makes sense.

rugabunda commented 5 years ago

Ok he updated cookiebro, now it clears SW.

verlain3 commented 5 years ago

It never crossed my mind to do that (what @Thorin-Oakenpants mentioned, about using browser built-in cookie block all by default) so i decided to give it a go and it seems to work just fine. However i feel like websites are not loading the way they used to, its like they are loading (or in this case deleting) more than just cookies and taking a sec or two more to load.

Can this be placebo? Or does this in fact deletes/blocks more than just the cookies?

rugabunda commented 5 years ago

Oh I see. Blocking cookies blocks localstorage, and SW also! News to me. You were right! It seems to be working perfectly.

rugabunda commented 5 years ago

@verlain3 it seemed to me like youtube took longer to load upon re-loading the webpage but I'm not sure if it is placebo either.. I'll have to test it out a bit. I will try Web developer network reload speed test.

curiosity-seeker commented 5 years ago

@rugabunda : You might want to look into the Forget Me Not add-on which offers as one cleanup type:

Instantly: Prevent data from being set, if possible. Otherwise behave like On Leave.

which Cookiebro does not. Besides, it also open source.

rugabunda commented 5 years ago

It never crossed my mind to do that (what @Thorin-Oakenpants mentioned, about using browser built-in cookie block all by default) so i decided to give it a go and it seems to work just fine. However i feel like websites are not loading the way they used to, its like they are loading (or in this case deleting) more than just cookies and taking a sec or two more to load.

Can this be placebo? Or does this in fact deletes/blocks more than just the cookies?

I notice it blocks a lot of cdn content, many java scripts that are designed to speed up the loading of various web components, such as cloudflares rocket-loader https://blog.cloudflare.com/we-have-lift-off-rocket-loader-ga-is-mobile/

You have to debug and see what java is being blocked and what each individual script does in order to know where you are breaking functionality or slowing your experience with storage/cookies blocked. Whitelisting cdns may be a good idea.

ajax.googleapis.com ajax.aspnetcdn.com ajax.microsoft.com dnjs.cloudflare.com ajax.cloudflare.com (rocket-loader) code.jquery.com cdn.jsdelivr.net yastatic.net yandex.st apps.bdimg.com libs.baidu.com lib.sinaapp.com upcdn.b0.upaiyun.com cdn.bootcss.com sdn.geekzu.org ajax.proxy.ustclug.org

I use privacy badger on top to make sure it filters out any tracking cookies and whatnot.

@curiosity-seeker, thank you for the suggestion, I just tried that yesterday and while it does offer an option to delete local storage on domain change, it didn't seem to work properly... it removed cookies but not local storage. I am actually noticing the same issue removing localstorage with cookiebro as well, when clicking "clear localstorage" only a restart of the browser fixes this.

rugabunda commented 5 years ago

So you get a huge amount of privacy and security blocking all local storage, however I found an interesting phenomenon when blocking all storage, privacybadger does not learn to block third party "domains". I have about 512 domains automatically blocked by privacy badger... that happened on top of ubo. Without privacy badger, these domains will be connected to and files downloaded, though not saved into "storage". You can use a pre-trained privacy badger profile on top of the local storage block to have even more privacy, but badger will not learn new things. So blocking third party with Ublock medium/hard mode sounds like the only option... possibly more than I am willing to go through.

rugabunda commented 5 years ago

@Thorin-Oakenpants this setting disables localstorage but uses sessionstorage, which requires a reboot of the browser to clear. So using this for a VPN is still not sufficient without rebooting the browser to clear sessionstorage.

imposing the following restrictions:

Cookies:

Block Cookie request headers and ignore Set-Cookie response headers.
Return an empty string for calls to Document.cookie and ignore requests to set cookies via Document.cookie.

DOM Storage:

localStorage: Window.localStorage is null. Thus, attempts to read and write using this object will throw a TypeError exception.
sessionStorage: read and write attempts are permitted.
IndexedDB: read and write attempts throw a SecurityError exception. 

Messaging and Workers:

Broadcast Channel: attempts to create a new BroadcastChannel will throw a SecurityError exception.
Shared Worker: attempts to create a new SharedWorker will throw a SecurityError exception.
Service Worker: attempts to create a new ServiceWorker will throw a SecurityError exception.

DOM Cache:

Calls to CacheStorage will always reject with a SecurityError.

Browser caches:

The HTTP cache and the Image cache are partitioned for tracking resources, such that each top-level origin will have a separate partition and tracking resources on different top-level origins will be cached separate from each other.

Network connections:

TLS sessions will not be resumed using a session ticket when an HTTPS connection is made to an embedded third-party resource that is classified as a tracker.
HTTP connection reuse by domains classified as trackers is limited to requests that occur under the same top-level origin. For example, a request for content from tracker.example on news.example will not reuse an HTTP connection with a request for content from tracker.example on shopping.example or with requests that occur when tracker.example is visited directly (i.e., as a first party).
rugabunda commented 5 years ago

I pulled the data from here, this is exclusive to the tracking protection setting: https://developer.mozilla.org/en-US/docs/Mozilla/Firefox/Privacy/Storage_access_policy#Tracking_protection_explained Ghacks says this is tracker blocking policy #4 which is blocking trackers by default... https://www.ghacks.net/2018/09/23/firefox-65-new-cookie-jar-policy-to-block-tracking/

Just got this email from Eff.org:

"I suggest keeping Firefox settings at their defaults, and letting Privacy Badger learn from cookies/local storage.

By sticking to defaults, you end up using the most tested and supported combination of browser and extension settings. (If Firefox were to eventually change its defaults to block all third-party cookies, that will be OK too, as we continue to add new kinds of tracking for Privacy Badger to learn from.)

Also, as you pointed out, blocking the domain entirely is more thorough than blocking cookies (or local storage) alone, as the domain may still track you by your IP address or your browser fingerprint."

As I had mentioned previously disabling localstorage results in badger no longer blacklisting domains, 512 of which were automatically blacklisted over time... I don't want to micromanage every little aspect of hundreds of websites with UBO at this stage.

rugabunda commented 5 years ago

Update, I believe I was right. I just noticed service workers appearing while I was browsing blocking third-party tracking... the only time these are disabled are with cookies blocked in total... that follows the template I shared above.

Thorin-Oakenpants commented 5 years ago

service workers are also blocked when cookies are set to session-only. Read this: https://github.com/ghacksuserjs/ghacks-user.js/blob/master/user.js#L1245. Go test on TZP

atomGit commented 5 years ago
  1. Cookiebro can automatically or manually remove IndexedDB, ...

yes, but auto happens only on browser start - the only ext. i found that automatically cleans IDB on a per-domain basis is Site Bleacher (it cleans other junk as well, such as workers)

personally i allow all 1st party cookies and let Bleacher deal with them because that's one less hassle (broken site functionality) for me, but i also disable disk cache, enable FPI, and run my profile in RAM

i also have issues with Nodetics (Cookiebro) and their goofy (IMO) restrictive licensing - if your goal is to help people, then GPL the code, publish it, and let others freely build on it - there's zero reason to do otherwise IMO, other than ego

nodeticswww commented 5 years ago

@atomGit it's not possible to delete indexedDB just for a particular domain on Firefox using WebExtension APIs. The reason is that Firefox doesn't support indexedDB.databases() and chrome.browsingData.remove() "hostnames" RemovalOptions parameter only affects cookies and localStorage (not indexedDB). See https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/browsingData/RemovalOptions

atomGit commented 5 years ago

i'm aware of the limitation of the webext api but Site Bleacher handles IDB in a novel way - try it yourself and you'll find that IDB storage is deleted upon revisiting a domain (as opposed to closing the domain tab)

nodeticswww commented 5 years ago

If the API specs are correct, it deletes IndexedDBs for all domains, not just the one for that particular domain.

On Wed, Aug 28, 2019 at 3:17 PM atomGit notifications@github.com wrote:

i'm aware of the limitation of the webext api but Site Bleacher handles IDB in a novel way - try it yourself and you'll find that IDB storage is deleted upon revisiting a domain (as opposed to closing the domain tab)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ghacksuserjs/ghacks-user.js/issues/776?email_source=notifications&email_token=AH345LUQDDPMKTCH26MO6MLQGZUETA5CNFSM4INNVZHKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5K5JPY#issuecomment-525718719, or mute the thread https://github.com/notifications/unsubscribe-auth/AH345LU7UIYFFGGLPLA6NUTQGZUETANCNFSM4INNVZHA .

crssi commented 5 years ago

@nodeticswww I know about API, but the author found a way around. Just try out Site Bleacher and see yourself.

I do not use those type of extensions, since TC in Auto mode perfectly deals with those pests. But if I would, then the Site Bleacher would won.

nodeticswww commented 5 years ago

See the code yourself: https://github.com/wooque/site-bleacher/blob/master/background.js - starting on line 102. It uses the standard chrome.browsingData.remove() with hostnames option but that shouldn't affect indexedDB purge settings. So if it does, the API specs are incorrect. I might as well test it with code.

Point being: if you have two sites open foobar.com and snafu.com and both have created an indexedDB database, calling that API will remove indexedDB on both (according to specs).

atomGit commented 5 years ago

test it yourself - it works, per domain, not global - you can use the webdev tools to inspect storage

again, it doesn't work in an expected way - it uses some JS that clears IDB upon revisiting the domain, so i'm guessing it isn't relying on the storage api at all, but rather working around it

looking at the code wouldn't do me much good since i'm not really a coder, but the SB dev told me how it works and it's been confirmed by others here that it indeed does work, crssi being one of them

nodeticswww commented 5 years ago

OK. I did some testing. Opened two websites: youtube.com and stackoverflow.com and created an IndexedDB database on both and added an entry of data in each database. Then I called the chrome.browsingData.remove() function with hostnames RemoveOptions parameter set just for one of the domains (e.g. youtube.com). As a result, IndexedDB on BOTH sites was emptied.

For some reason you don't seem to understand that while IndexedDB is removed on one site, it is ALSO removed on ALL other sites as well which may break website functionality "in-flight".

atomGit commented 5 years ago

test:

nodeticswww commented 5 years ago

Huh? cnn.com doesn't even use IndexedDB.

atomGit commented 5 years ago

yeah, i think i'm not testing correctly due to goofiness in the way the dev tools/storage refresh option works - it doesn't actually refresh the list items - appears you have to actually close and re-open the tools to update the list

when i loaded cnn.com, a list of domains was under idb storage, all of which don't seem to show anything was actually stored other than a "folder" named 'test', but the same was true for youtube.com - i just saw a list of domains with no data - i have to fiddle around to get youtube to store IDB it seems???

i didn't use stackoverflow because the result for me was the same as youtube - a list of domains, all with 0 storage apparently

i'll hit this again and see what happens

atomGit commented 5 years ago

i did the test again using stackoverflow and youtube and it was exactly the same as my test above

the storage for stack was the same as in my cnn test case - just a list of domains, one folder named '1', no data stored for any of those "keys" i assume they're called

closed and reopened stackoverflow and the youtube idb was unaffected

nodeticswww commented 5 years ago

@atomGit you are not testing it properly. Please be aware that Firefox has a bug that doesn't properly update the Storage view after e.g. indexedDB databases have been deleted unless you close and reopen the devtools panel.

Do this:

  1. Install Site Bleacher unless already installed.
  2. Open youtube.com and open devtools panel and Storage tab. Ensure you see "swpushnotificationsdb" database there. Leave the tab open.
  3. Now open stackoverflow.com in another tab (and ensure it's not whitelisted).
  4. Close the stackoverflow.com tab.
  5. Go back to your open youtube.com tab and close the devtools panel.
  6. Open the devtools panel again and open the Storage tab. See under Indexed DB that the database is now gone.

This is the last time I comment on this issue. I'm tired of repeating myself.

Luckily this issue got just assigned: https://bugzilla.mozilla.org/show_bug.cgi?id=934640

On Wed, Aug 28, 2019 at 6:19 PM Thorin-Oakenpants notifications@github.com wrote:

For some reason you don't seem to understand that while IndexedDB is removed on one site, it is ALSO removed on ALL other sites as well which may break website functionality "in-flight".

I'm not doing any testing, but FPI should solve this issue

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ghacksuserjs/ghacks-user.js/issues/776?email_source=notifications&email_token=AH345LUZM4BMNYEOQ4EB3PTQG2JO7A5CNFSM4INNVZHKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5LPQEI#issuecomment-525793297, or mute the thread https://github.com/notifications/unsubscribe-auth/AH345LS5H4FKPNTW4OFSJ73QG2JO7ANCNFSM4INNVZHA .

atomGit commented 5 years ago

i tested exactly as you describe before, but i just repeated the test again precisely as you outlined, steps 1 - 6, and again the youtube idb storage was unaffected - i'm running 68.0.1 on linux

i don't know why we're seeing different results, but there it is

anyway, glad to see the idb storage enumeration thingy is FINALLY being addressed ... now for the CSP issue (not holding breath)

rugabunda commented 5 years ago

Just tested site bleacher in windows, latest firefox, indexdb was cleared for all tabs. It successfully removed localstorage and indexedDB upon closing and reloading TZP-sanitizing however javascript 1st party persistent cookies remain until blacklisting github.io

javascript 1st party persistent cookie [zombie TZP cookie found] value: 2x6dg27l48g localStorage [nothing found] setting new zombie TZP key: value 2x6dg27l48g indexedDB [nothing found] setting new zombie TZP store: value 2x6dg27l48g appCache todo service worker cache todo

cleared everything properly in another firefox browser... not this one, due to the git container.

atomGit commented 5 years ago

I'm not doing any testing, but FPI should solve this issue

i totally missed the importance of pants' statement - this is perhaps why we're getting different results with Site Bleacher - i always enable FPI - however this doesn't explain why IDB storage doesn't seem to be affected for a second domain (even after reloading dev tools) in my case using a fresh profile where FPI is disabled by default

nodeticswww commented 5 years ago

I have to add that FPI has no effect on chrome.browsingData.remove() behaviour.

On Thu, Aug 29, 2019 at 2:50 PM atomGit notifications@github.com wrote:

I'm not doing any testing, but FPI should solve this issue

i totally missed the importance of pants' statement - this is perhaps why we're getting different results with Site Bleacher - i always enable FPI - however this doesn't explain why IDB storage doesn't seem to be affected for a second domain (even after reloading dev tools) in my case using a fresh profile where FPI is disabled by default

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ghacksuserjs/ghacks-user.js/issues/776?email_source=notifications&email_token=AH345LSSFEG6KGJBW3WQH33QG6ZW5A5CNFSM4INNVZHKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5OGXMQ#issuecomment-526150578, or mute the thread https://github.com/notifications/unsubscribe-auth/AH345LT5ZKRO5N36JZDOCNTQG6ZW5ANCNFSM4INNVZHA .