search

Lusito / forget-me-not

Make the browser forget website data, except for the data you want to keep.
zlib License
227 stars 25 forks source link

IndexedDB scrubbing is broken #136

Open Joel889 opened 5 years ago

Joel889 commented 5 years ago

1) Visit http://jsfiddle.net/unclelongmao/VrS32/ 2) IndexedDB "http+++fiddle.jshell.net" is created in /storage/default 3) "Clean this domain" 4) Perform "on demand cleaning" (with all checkboxes selected) 5) Entry is not getting deleted. 6) Enable IndexedDB clean on restart (make sure FF's internal cleaning on shutdown settings are disabled) 7) Restart browser 8) Entry is not getting deleted.

Lusito commented 5 years ago

There is something fishy going on here, but it doesn't seem to have anything to do with FMN. Running the browsingdata.remove() api manually doesn't clean the folder either. Weird thing is, that sometimes the cleanup causes the todo list to empty and sometimes it doesn't. I will investigate further tomorrow. Maybe it has to do with #115.

Lusito commented 5 years ago

Created an issue at mozilla for this: https://bugzilla.mozilla.org/show_bug.cgi?id=1529122 and #115. Not sure if I can fix this at my end.

Joel889 commented 5 years ago

Thank you for looking into it. I'm assuming the API is broken, because FF has no issue deleting all IndexedDB's automatically on startup (if set in FF's privacy settings).

streetwolf commented 5 years ago

Just as a point of interest, the cookie manager in Fx Nightly (maybe the stable version too) lists these entries (site data) and let's you delete them. So does CCleaner after Fx closes. Have you tried cleaning them under the latest Nightly version for Windows?

Lusito commented 5 years ago

As far as I can tell, this is a web-extension specific issue. Firefox itself is able to clear it with its UI, but web-extensions access a different method to clear the data.

streetwolf commented 5 years ago

Here’s hoping that one day it will work either by Mozilla allowing it or you finding a work around.

Gary

From: Lusito notifications@github.com Sent: Wednesday, March 13, 2019 2:48:28 PM To: Lusito/forget-me-not Cc: Gary; Comment Subject: Re: [Lusito/forget-me-not] IndexedDB scrubbing is broken (#136)

As far as I can tell, this is a web-extension specific issue. Firefox itself is able to clear it with its UI, but web-extensions access a different method to clear the data.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/Lusito/forget-me-not/issues/136#issuecomment-472556474, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AAZErw1l7KwRa8SyqyPZSHxqfsVD9uf8ks5vWUf8gaJpZM4bBoOf.

Lusito commented 5 years ago

@Joel889 @streetwolf It seems, mozilla is on it right now. If you can supply error messages or any more detail, please do so.

Lusito commented 5 years ago

https://bugzilla.mozilla.org/show_bug.cgi?id=1526246 seems to be part of the issue.

OhSoGood commented 5 years ago

@Lusito : any news about this entry? Some sites I visit intensively uses DB to track users... Issue 1529122 on Firefox is closed now, so could it be fixed? For 1526246, I don't feel so concerned as services workers are disabled on my firefox - am I right?

Lusito commented 5 years ago

1529122 is fixed in Firefox 68. Latest official release is 67. So unless you are using the beta/developer/nightly version, you'll still have to wait a bit before you can try it.

OhSoGood commented 5 years ago

Hi @Lusito , any updates on this since FF 68 is out for a while now? If I can help by testing, please ask!

vavavr00m commented 5 years ago

Would appreciate a workaround for forks and/or older versions of FF. Site Bleacher works while neither manual and instant cleanup (FMN) works on Waterfox Current 2019.10 (64-bit).

Lusito commented 5 years ago

There is no workaround for this. It needs to be fixed by the browser.

If you can really reproduce it, that Site Bleacher works and FMN does not, please tell me the steps to reproduce and I will take a look at what is different between the two extensions.

But last time I investigated Site Bleachers code, I noticed, that it does not work as promised on Firefox. They wrote it for chrome and assumed the same things work on Firefox, but on Firefox there are some differences in the API making things not work as Site Bleacher promises. For example, the per-domain cleanup on Chrome supports cookies, cache, and storage (CacheStorage, FileSystems, IndexedDB, LocalStorage, PluginData, ServiceWorkers, and WebSQL). While Firefox only supports this for cookies and local storage. In these cases, the cleanup on SiteBleacher would apply to all domains rather than just the one domain the user intended.

vavavr00m commented 5 years ago

Site Bleacher

  1. Install Site Bleacher
  2. Go to http://jsfiddle.net/unclelongmao/VrS32/ and IndexedDB "http+++fiddle.jshell.net" is created in /storage/default
  3. Close http://jsfiddle.net/unclelongmao/VrS32/ tab
  4. IndexedDB folder "http+++fiddle.jshell.net" is deleted

Forget-Me-Not

  1. Install Forget-Me-Not
  2. Enable all rules for all options (startup, on leave, instantly)
  3. Set Delay for domain leave cleanup in seconds to 0
  4. Go to http://jsfiddle.net/unclelongmao/VrS32/ and IndexedDB "http+++fiddle.jshell.net" is created in /storage/default
  5. Click Clear this domain button (trash button works though)
  6. IndexedDB folder "http+++fiddle.jshell.net" is not deleted
  7. Create a rule to cleanup on leave or instantly
  8. Close http://jsfiddle.net/unclelongmao/VrS32/ tab
  9. IndexedDB folder "http+++fiddle.jshell.net" is not deleted

You may say it was explained but it is a bit vague " For On Leave and On Startup to work, the respective cleanup options must be enabled. Prevent data from being set, if possible. Otherwise behave like On Leave. Some limitations might apply due to restrictions in the Web-Extension API**. Read more" What are these restrictions? Clicking read more didn't explain which data was not being cleaned or my comprehension is bad. Taking Cleanable Data list out of the settings and moving it below the cleanup rules would make more sense to me. Instead of purely Global rules, there could be per-domain / sub-domain cleanup rules as well.

Mock-up

I hope it made sense to you.

samr7 commented 4 years ago

I'm seeing the same behavior as vavavr00m using FMN 2.2.5. Visiting http://lusito.info/storage/, running FMN cleanup, and reloading:

Cookie:I don't remember you Local Storage:I don't remember you IndexedDB:I remember you

Same process with Site Bleacher 1.2.1:

Cookie:I remember you Local Storage:I don't remember you IndexedDB:I don't remember you

Lusito commented 4 years ago

IndexedDB can not be cleared on domain leave, because it can't be cleared per-domain. See (and vote for) this issue: https://bugzilla.mozilla.org/show_bug.cgi?id=1551301

That's also why the "clear this domain" does not clean indexeddb of the active domain, since it would have to delete indexeddb of all domains, which is not what the user would expect.

What SiteBleacher does is clean all IndexedDB. Not just the one from the page you are leaving. Cleanup on startup and manual cleanup should work (but it cleans all domains).

You can confirm this by using two websites that have indexeddb. For example:

Leave one with SiteBleacher and the other one will be cleaned as well.

jcs commented 4 years ago

Looks like this will be in Firefox 77

https://hg.mozilla.org/mozilla-central/rev/d922e7e833ba

Lusito commented 4 years ago

yes, the domain-specific cleaning for indexedDB and service workers will be in FMN V3. This issue, however is not about domain-specific cleaning. It only went off-topic.

Ileca commented 4 years ago

Do we have some update on this because it's been a year and I still have tons of junk that refuse to be deleted (both stuff that appears in the Cookies and Site Data manager and stuff that doesn't appear there because it's only a ".metadata-v2" file, which is apparently not enough to be listed).

It grows and you wonder what's the point of having an add-on which is supposed to clean site data if the API is completely broken. You would think that such a privacy issue would get a privileged treatment from a privacy-conscious browser but apparently not.

This is quite frustrating...

OhSoGood commented 4 years ago

@Ileca : a long-time user of ForgetMeNot and alikes, I've finally switched to another solution which is more efficient in my eyes and with my needs: temporary containers. Any new tab or so gets opened in a dedicated temporary container, and that container gets deleted when all tabs belonging to it are closed, thus effectively removing any cookie, DB, whatsoever. Additionally, there are settings, rules and companion extensions to customize the behaviors (ex: to have some site which keep their data, etc.). The link: https://addons.mozilla.org/firefox/addon/temporary-containers/ Enjoy!

Ileca commented 4 years ago

I used containers (firefox containers and temporary containers addons) for a while but it was way too cumbersome (I don't want a new tab each time I have linked a website to a specific container) and has a big weakness: you can't whitelist websites for a container, meaning that if I open a container for a website I trust and open a facebook link in it, it will be contaminated by facebook crap. I also have to create new containers each time I want to be able to connect to a website in the long run and link it to the container, it's an annoying process which, like I said, fail once you click on outside links in it. That's why I prefer whitelisting "trusted" site data, allow some more with uMatrix depending on the need and wait on Forget-me-not to clean the mess afterwards.

Lusito commented 4 years ago

@Ileca this is partially solved as you can see by the status of the mozilla tickets.

I'm not sure why Mozilla works more on making the searchbar uglier with each release than fixing some of the privacy issues. I'd love to contribute to the code myself, but there is a very high entry barrier with their tooling, which would take me a lot of time to just be able to start working on something. I don't have that amount of time right now.

I feel your frustration though.

@OhSoGood Temporary Container is another approach, but it has its own drawbacks. If temporary containers work for you, then by all means, use them! They don't work on android though.

Forget Me Not is about choice of what you want to keep. Other solutions might fit your specific use-case better. For other use-cases, FMN might be the best option, even if some things are broken in the browser API.

OhSoGood commented 4 years ago

@Lusito : I was sad to drop Forget-me-not and indeed it's not your fault but Mozilla's one. That impossibility at the browser level for extensions to targets databases is a major drawback, especially when we see all these websites moving to use DBs. I liked a lot your extension and, if/when Mozilla decide to move themselves a little, I may well come back from temp containers to forget-me-not. Thank you a lot for your work and time, I know as an opensource maintainer what that means.

streetwolf commented 4 years ago

I cleared my cache and cookies and also deleted webappsstore.sqlite. I can now delete local storage on browser restart. Doing all of this also fixed some other issues I had with some sites not related to your extension. So you can close my ticket.

Regards,

Gary

From: Gary Shapiromailto:garyshap@outlook.com Sent: Wednesday, July 22, 2020 2꞉17 PM To: Lusito/forget-me-notmailto:reply@reply.github.com Subject: RE: [Lusito/forget-me-not] First web site opens very slowly with FMN enabled (#242)

Windows 10 x64 1909. Like I said let me try a new profile again and see what happens.

Regards,

Gary

From: nerouxmailto:notifications@github.com Sent: Wednesday, July 22, 2020 2꞉16 PM To: Lusito/forget-me-notmailto:forget-me-not@noreply.github.com Cc: Garymailto:garyshap@outlook.com; Authormailto:author@noreply.github.com Subject: Re: [Lusito/forget-me-not] First web site opens very slowly with FMN enabled (#242)

Then I am out of ideas. Doesn't happen here. Which operating system?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/Lusito/forget-me-not/issues/242#issuecomment-662607975, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AADEJLZ452HST4DJQ6CCDQDR44UGZANCNFSM4OREFRTQ.