search

arkenfox / user.js

Firefox privacy, security and anti-tracking: a comprehensive user.js template for configuration and hardening
MIT License
9.97k stars 511 forks source link

Can't set exceptions for cookies [solved: use FPI syntax + don't sanitize cookies on close in 2803] #1223

Closed Dupond closed 3 years ago

Dupond commented 3 years ago

Hello,

I'm wondering if I'm doing something bad, but I can't set exceptions for cookies with the current user.js file. Or is it intended?

I've read the wiki, so I know that "We delete cookies and history (download, form and browsing histories) on close"; so I'll try to explain what I don't understand exactly (sorry for my bad english): wouldn't it be better to set network.cookie.lifetimePolicy to 2 (this pref is commented in the user.js file for the moment) and privacy.clearOnShutdown.cookies to false, so that cookies would be deleted when Firefox is closed, but we could still set exceptions?

The current behaviour is: you can set an exception for cookies, but anyway all the cookies are deleted when Firefox is closed, so any exception is simply useless.

I'm surely missing something here, and hope I don't bother anyone. Thanx for helping me, Best regards, Dupond

Dupond commented 3 years ago

Thanks for the very detailed answer! But obviously I haven't explained myself well... but I haven't found how I could reopen, so I'm just leaving another comment hoping you'll see it.

So here's what happens: I use a default user.js file (so FPI is enabled, etc.); then I do exactly what you do in your "Answer 1" (Ctrl+i > Permissions > Set cookies > check "Allow") and have a correct exception set (the website includes the ^firstPartyDomain=... part, the cookie is set, everything is fine). But since all cookies are deleted when Firefox closes (because of the privacy.clearOnShutdown.cookies set to true in the user.js file) the cookie for the allowed website is deleted too when I close Firefox. You seem to suggest that it's not the case for you, and that's what I don't understand.

Note that the exception is (of course) still set after a Firefox restart, but it's simply useless: all the cookies (including ones for which I set an exception) are deleted when Firefox closes...

What am I missing here?

rusty-snake commented 3 years ago

Answer 2:

Sanitizing on close removes everything […], you can't have exceptions […]

If you want to remove all cookies except for a few, configure FF to keep all cookies and use TC in auto-mode or reject all cookies except for the few you want to keep.

Thorin-Oakenpants commented 3 years ago

see 1681701, spun off from 1681493#c1

Dupond commented 3 years ago

I must be a complete idiot, since all these answers are still not 100% clear for me: so, just to be sure I understand well, my final question is: was I right to assume that with the current user.js, setting an exception is useless, since even when using FPI syntax, all cookies are deleted when I close Firefox?

Thorin-Oakenpants commented 3 years ago

so, just to be sure I understand

Clearing on shutdown (2803) is ALL or NOTHING - it was never designed to sanitize by host, or domain, or eTLD+1, or by origin attributes, and don't forget about scheme - it does this by destroying the cookies.sqlite file, or permissions.sqlite, or bulk clearing the IDB folders etc = all fast and easy (or something like that, I'm not actually going to look at the code). This section has been around since 1066 and was commissioned by William the Conqueror. It is a very blunt instrument and everything got stuck under a history section

Since then, around about when ETP was introduced, they created a Cookies + Site Data section, they created Quota Management v3 (to tie all the site data into one lump and control it - v4 is on its way next), they created LSNG (for reliability), and they allowed users to now selectively keep some site data on close (via exceptions), because it's all wired up to handle all quota management data by host, or domain, or eTLD+1, by origin attributes, scheme .. etc.

This newer Cookies + Site Data mechanism currently uses cookie lifetime as the controlling pref, by using "session-only" - but "session-only" is a cookie attribute, not an IDB or service worker cache etc attribute.This is at least one reason why they want to remove it, as setting a cookie attribute has nothing to do with ALL site data. I do know that early on there was a bug that if a "cookie" wasn't set, the sanitizing didn't know that domain was "session-only". I suspect maybe what they want to do is put site data that isn't meant to be kept, into memory only - but that has issues with at least IDB (which is why currently IDB is not supported in PB mode - there is no mechanism for memory only IDB right now - but they're working on it - encrypted and in memory only), and probably service workers

IANAE on this, but there's a lot of moving parts and it's very complicated (especially when you factor in session restore and Forget about this site, and orthogonal items like history, download and form entries). Open bugzillas on sanitizing, when I last looked, was over 500 open bugs. Even the devs at times don't know what's going on - see 1681493#c2 - "To be honest I didn’t realize that this wasn’t the case already"

At some stage, the history section sanitizing of cookies (2803, cookies) + IDB (2803, offlineApps) should get removed IMO - and solely handled in the cookies + site data section - and by that stage the lifetime pref probably won't still be around

Edit: it helps that appCache is no longer supported

was I right to assume that with the current user.js, setting an exception is useless

No. Exceptions are applied when it doesn't match the default you set. If the default is to allow, you can still set block or allow for session. For me, I block by default, and thus set exceptions as allow or allow for session

Thorin-Oakenpants commented 3 years ago

@githubuniqu Android is a different kettle of 🐟 ... since closing an app doesn't shut it down

Dupond commented 3 years ago

No. Exceptions are applied when it doesn't match the default you set. If the default is to allow, you can still set block or allow for session. For me, I block by default, and thus set exceptions as allow or allow for session

Well, that's not what I see here, and that's why I don't understand since the beginning:

rusty-snake commented 3 years ago

Exceptions is NOT about deleting

Clearing on shutdown (2803) is ALL or NOTHING - it was never designed to sanitize by host, or domain, or eTLD+1, or by origin attributes, and don't forget about scheme - it does this by destroying the cookies.sqlite file, or permissions.sqlite, or bulk clearing the IDB folders etc = all fast and easy (or something like that, I'm not actually going to look at the code).

Exceptions is about allow/block

Dupond commented 3 years ago

I got that; but Thorin seems (to me) to suggest that he can set some exceptions so that his cookies are NOT deleted when he closes and reopens Firefox...

Since everyone has been more than patient here, I think I'll give up for now, since obviously my english understanding is far from being as good as I thought, and I must be missing something. And above all I'm concerned to feel constantly like a little hobbit, lost in the forest, and praying for some help from the Great Soldier... on a (rainy) sunday!

Thanx a lot everyone for all the explanations; I'll come back here in the next days in order to read all this again and try to improve my understanding of what happens, and why.

Best regards, Dupond

Thorin-Oakenpants commented 3 years ago
  • unmodified user.js in vanilla profile
  • change network.cookie.cookieBehavior to 2
  • add "cookie" exception for test site as "Allow"
  • close Firefox, reopen it
  • check cookies (don't visit the site) - the cookie is GONE ... also all other data is gone like IDB, service worker cache etc - why?

Because you are still using 

//     - "Offline Website Data" includes appCache (2730), localStorage (2720),
//       service worker cache (2740), and QuotaManager (IndexedDB, asm-cache)

user_pref("privacy.clearOnShutdown.cookies", true);
user_pref("privacy.clearOnShutdown.offlineApps", true); // Offline Website Data

Firefox is doing what you told it to do

You need to change privacy.clearOnShutdown.cookies to false : and if you want to keep any other site data, also change privacy.clearOnShutdown.offlineApps to false

Thorin-Oakenpants commented 3 years ago

I got that; but Thorin seems (to me) to suggest that he can set some exceptions so that his cookies are NOT deleted when he closes and reopens Firefox...

Not sure what gave you that impression. I was talking about setting exceptions in my example. I never anywhere said items were retained. I block all by default, but allow four, and allow for session on another dozen of whatever

As it is, I do retain those four, because I have privacy.clearOnShutdown.cookies = false in my overrides, I still clear privacy.clearOnShutdown.offlineApps. But I never said I did that.

So you are the same. Instead of block all by default, you are session only by default. You add some sites as Allow exceptions, flip the sanitize on close pref like I did, and voila, you keep those Allow cookies between restarts

capisce? (that's Italian for "Do you understand?")

Thorin-Oakenpants commented 3 years ago

I added an override recipe for it

Dupond commented 3 years ago

Capisco! Thanx a lot for the last commits. Things are much clearer for me now.

The way I was dealing with it (network.cookie.lifetimePolicy set to 2 and privacy.clearOnShutdown.cookies set to false) allowed me to use uMatrix to easily deal with cookies, whereas if I block them all by default and add exceptions, I'll have to create dozens of them, and it's far from being as practical as uMatrix (one click to allow, one click to block); and if I stay with the defaults (only block 3rd party cookies) and doesn't clear cookies on shutdown, then I'll quickly have hundreds of cookies from hundreds of sites...

So now I guess I'm stuck with:

In any case, I will miss my current settings; they were so easy to deal with! And with the last uMatrix update by gorhill, I was hoping that I could keep them for at least a few years more... For now, I'll keep my configuration, as long as network.cookie.lifetimePolicy is not yet deprecated.

Thorin-Oakenpants commented 3 years ago

allowed me to use uMatrix to easily deal with cookies

⚠️ uMatrix does not fully block cookies - it only blocks cookies being read back via request headers - JS can bypass it. Do not use uMatrix as a cookie blocking strategy

do nothing and hope

They will deprecate the lifetime pref, but the setting to "delete all cookies + site data on close (with exceptions)" will remain - I just don't know what form that will take. You won't lose any functionality

and doesn't clear cookies on shutdown

no need for that. Either wipe all (clearonshutdown prefs) or don't wipe all and use lifetime = 2 (with allow exceptions)

whereas if I block them all by default and add exceptions

Works for me, but everyone has different mileage. At some stage this will become unfeasible: FPI will be removed and dFPI (which allows all cookies except tracker ones) will be our default, the cookieBehavior pref (2701) can't be both 1 and 5

uBO can block all 3rd party by default (that's what I do), so the only thing I will miss is the fact that auto blocking any web storage just makes everything so much cleaner and less busy having to deal with cleaning up afterwards

Sanitizing in-session is a bit pointless. You still have your IP address and other vectors (SSL session ids, etc) that can link repeat visits to a 1st party site (not saying it's totally useless - e.g. "Forget this Site" to continue to bypassing a paywall: not all sites are the same). Browsers weren't designed to do this - even Tor Browser doesn't wipe all site data if you request a new circuit, it's not meant to, that's what New Identity is for. New Identity is a whole new session. Temporary Containers in hardened mode is almost there, it just can't directly handle the IP issue.

Your protection in-session is FPI (or dFPI) and the key is sanitizing between sessions - e.g. when all PB mode windows are closed, all PB mode memory is released, or if FF is closed, all memory and specified disk data is wiped.

Dupond commented 3 years ago

Do not use uMatrix as a cookie blocking strategy

But would there be a real risk, though? Considering that I use FPI with 3rd party cookies blocked (and all the other default settings from the user.js file, except for privacy.clearOnShutdown.cookies that I set to false) + uMatrix blocks (or, as you noticed, retains) all cookies by default, with exceptions for the websites on which I want to be able to login + all cookies are deleted when I close Firefox (network.cookie.lifetimePolicy set to 2)... What bad could happen regarding cookies, then? Furthermore, I close and open Firefox quite frequently...

Thorin-Oakenpants commented 3 years ago

Considering that I use FPI

FPI protects against cross site tracking

But would there be a real risk, though

depends on how you define risk, and what you are trying to achieve. With FPI/dFPI you don't need to worry about cross site tracking - that leaves 1st party tracking, and until you mask/change your IP frequently, don't waste your time.

uMatrix's cookie setting doesn't block cookies being set, it thus does not block any site data (localStorage, sessionStorage, IDB, service worker cache). It's sole purpose is to block sites reading back values via response headers = not a full or proper solution: I can still read cookies via JS, I can still set and read IDB, localStorage etc. Complete waste of time IMO.

So, again, what are you trying to achieve? Answer this question, and you have your answer. If you say you want nothing to link you on repeat visits, then the answer is:

Like you, I have short FF sessions. I use my main FF all the time for short stretches (1 or 2 hrs max at a time), and also Nightly. I guess I open and close Nightly two or three dozen times a day, as I test changes or have ideas on fingerprinting etc. Nightly is vanilla with uBO (default just to kill ads) and a couple of tweaks e.g. dark mode, ETP strict mode, HTTPS-only mode, sanitize everything on close. Nightly is 95% for testing and some research.

That's it. If you want to change your "identity", you need a new session: sanitize on shutdown, it clears everything out like ssl session ids, state storage, etc. Just like Tor Browser has New Identity = a new session. Tor Browser has the IP covered, FF natively does not.

So again, what are you trying to achieve by blocking cookies? I do it because I can live with it and to me it means 99.9999% less CPU and disk usage and cleaning up - there's nothing to store, remember, write or clean

Thorin-Oakenpants commented 3 years ago

What bad could happen regarding cookies

toomanycookies

Thorin-Oakenpants commented 3 years ago

I suspect maybe what they want to do is put site data that isn't meant to be kept, into memory only - but that has issues with at least IDB (which is why currently IDB is not supported in PB mode - there is no mechanism for memory only IDB right now - but they're working on it - encrypted and in memory only), and probably service workers

Ahh, that's right, it was just encryption-on-disk and sanitize on close, so even a firefox crash still means the data is protected

The tentative plan is indeed to use temporary disk storage that's encrypted so that in the event of a browser crash the data will remain unreadable. We're starting with IndexedDB support for this mode of operation ...

practik commented 3 years ago

@Dupond & everyone else: Thanks for all the questions and answers. Reading this thread has given me a much better understanding of how some of this stuff works. It's super helpful to see how other people are approaching the problem.

@Thorin-Oakenpants : Did you link the wrong Bugzilla in your last comment? I don't see the connection …

Thorin-Oakenpants commented 3 years ago

Did you link the wrong Bugzilla in your last comment

I did indeed. Now fixed. The linked comment was quoted underneath, so you didn't miss much

Thorin-Oakenpants commented 1 year ago

FPI is no longer supported, and we do sanitize on close since exceptions are respected