[ISSUE] TC breaks Cache API (Local storage)

iWARR commented 4 years ago

Impacted: FF70 - FF75 (hard slowdownings are on FF70-FF74)
OS: Windows 10 / 8.1
FF75, fresh profile, completed setup (settings, add-ons, tweaks, etc.), ready for regular and usual use.
Multi-Account Containers (MAC) and Facebook Container are not affected (healthy).

[TEST] https://firefox-storage-test.glitch.me/

Debug Info


Storage is broken. This is your first visit or all storage was automatically cleared.
Specific Subsystem Statuses:

LocalStorage
    Good: Totally Working. (fullyOperational)
QuotaManager
    Good: Totally Working. (fullyOperational)
IndexedDB
    Good: Totally Working. (fullyOperational)
Cache API
    Bad: Our test logic is broken, please copy and paste the contents of 'Debug Info' below and anything in the devtools console and send to :asuth. (unexpectedBreakage)

Debug Info:

{
  "v": 1,
  "curVersion": 75,
  "prevVersion": 0,
  "ls": {},
  "qm": {
    "lastWorkedIn": 75
  },
  "idb": {
    "persistentCreatedIn": 75,
    "persistentLastOpenedIn": 75,
    "clearDetectedIn": 0
  },
  "cache": {
    "firstCacheCreatedIn": 0,
    "unpaddedOpaqueCreatedIn": 0,
    "paddedOpaqueCreatedIn": 0
  }
}

ISSUE and STEPS TO REPRODUCE

Before TC installation: the performance of FF is OK. Test passed. After installation TC + making first settings + restarting the browser: Test failed (Cache API: Bad)

Futhermore, TC hardly breaks FF cache

1) Loading of the Test webpage takes a very long time (NOT the test itself, just a webpage!) or can't load at all during several FF restarts / until TC uninstall.

2) Simple Disabling / Uninstallation of TC Add-on doesn't help: FF continue loading Testing page long time, test failed. Cache was broken by TC.

3) How to make Cache API healthy again: ---> TC Options -> Advanced -> Reset storage -> Wipe local storage and reset it to default ---> Probably (not confirmed) you'll want to delete "permanent" и "temporary" filders inside your profile: C:\Users\%USERNAME%\AppData\Roaming\Mozilla\Firefox\Profiles\XXXXXXXX.default-release\storage\ ---> Immidiately Uninstall TC and Restart FF

Test result : "Cache API" : Good | Loading of the testing webpage becomes faster.

Additional (possible) impacts (especially FF70-FF74 slowingdowns):

Every time FF loads first Tab with lag (any first webpage after start) or produces infinite page loading (endless loop). After refreshing the page the lag mostly gone (until the next FF start only).
Seems like this bug also slowing down other Add-ons that use Local Storage - uBO database, etc. This was tested on FF before v75 (with and without uBO)

That was extremely annoing since FF70 :(

PROOFS (Note: everywere people don't know who is guilty):

• Reddit

[1] IndexedDB & Cache API are bad and broken. How to fix? [2] Firefox takes about a minute to "connect" after startup

• Bugzilla

[3] The War on Storage Initialization Failures [4] chrome.storage.local is not ready on startup causing extension corruption/ infinite page loading [5] response_padding_size doesn't exist in cache.sqlite while accessing it in storage initialization [6] Numerous sites fail to load after upgrade to Firefox 70.0

• gHacks

[The real workaround here is (unfortunatelly) moving to the new profile only]

[7] Firefox 70 not loading some pages or elements? Here is a fix

Yesterday (AT LAST!) I've found that highly likely it was TC Add-on.

I'm sure the described issue mostly caused by unperfect FF solutions for the Local storage. Really, ugly FF devs work. Hovewer, at present time, on FF75 (that is better and faster than any previous v7X), I see this bug comes exactly from TC.

Please, discover this and fix the TC Add-on ASAP, because this is the one of the most powerful security/privacy components we have to counteract corporations in days of total surveillance.

And thank you very much for this cool addon.

stoically commented 4 years ago

Tested with Firefox Stable 75 and Beta 76.0b5 and the test page always returns "fullyOperational" for all caches on my end. Did you try with a new profile and no custom preferences?

In general, TC uses the exact same API call (there's only one) as MAC and FC - so if it breaks for TC, it should basically break for all container Add-ons.

crssi commented 4 years ago

@iWARR

Did you try with a new profile

^^ this

iWARR commented 4 years ago

EASIER WAY TO REPRODUCE THE "TC" BUG

You even don't need to Install/Uninstall "Temporary Containers" Addon. Please, backup (export) your TC settings before you start.

TEST-1

1) First Party Isolation (FPI) = [ Enabled ]

"privacy.firstparty.isolate" = true

2) Make Cache API healthy (clean the Local storage damage):

a) Turn OFF the checkbox: TC Options -> General -> [ ] Automatic mode b) TC Options -> Advanced -> Reset storage -> [Wipe local storage and reset it to default]

3) Restart FF

4) Turn ON the checkbox: TC Options -> General -> [•] Automatic mode

5) [TEST] https://firefox-storage-test.glitch.me/

Result: Cache API : Bad

TEST-2

1) First Party Isolation (FPI) = [ Disabled ]

"privacy.firstparty.isolate" = false

2) Make Cache API healthy (clean the Local storage damage):

a) Turn OFF the checkbox: TC Options -> General -> [ ] Automatic mode b) TC Options -> Advanced -> Reset storage -> [Wipe local storage and reset it to default]

3) Restart FF

4) Turn ON the checkbox: TC Options -> General -> [•] Automatic mode

5) [TEST] https://firefox-storage-test.glitch.me/

Result: Cache API : Good

crssi commented 4 years ago

Did you do the reproducable steps on a clean, brand new, plain vanilla firefox profile?

iWARR commented 4 years ago

I've done the new tests on completely new profile. Results are the same.

What was included (min FF setup):

• Policies (not affected) • MAC + Facebook containers (not affected) • uBO (not affected) • Custom profile: FPI = ON (affected) / OFF (not affected) • TC Addon (depends on FPI state)

crssi commented 4 years ago

Ditch all add-ons except TC and try again. Essentially go with a brand new profile + TC only and firefox prefs, no other addons.

iWARR commented 4 years ago

@crssi Then my data will leak to Mozilla. FF is not trusted software for me by default. I never use a plain unporected configs (at least, base setup should be done). I'm always must preconfigure FF offline before 1-t sart, then go. Sorry.

crssi commented 4 years ago

OMG. I meant for a test, not for a usage.

Also be thorough, what exactly does mean Policies?

stoically commented 4 years ago

@iWARR

TEST-1

Tried that. Gives me "Cache API: Good" on new profile with just TC installed and FPI enabled.

If it works with a vanilla profile with no custom about:config preferences then this isn't an TC issue.

iWARR commented 4 years ago

what exactly does mean Policies?

They mostly do the same as "user.js" through "cfg" - they lock most important prefs. In contrast to cfg (where you can lock any pref), the Policies are hard built-in FF preconfiguration, where nothing can override chosen presets at all. This is the the first milestone of your protection. They apply globally, for any new FF profile automatically. Then goes "user.js" add-ons and everything else.

See here: policy-templates

@stoically Then, probably, FPI is not the one here. May be some other custom preset have influence also. What I can say... My tests are well done, they clear and say smth. to us... But not everything, so far.

iWARR commented 4 years ago

@stoically If you wish, you may close this issue. When I will have some extra new info, I'll inform you and other hundreds of people expecting the same issues decribed in the PROOFS section.

Again, I beleive it's moslty Mozilla's failure. TC & FPI just demonstrate us (those, who was affected) the bug in "real deal".

stoically commented 4 years ago

If you like we can keep it open until you found the real source of the issue, would be interesting to know what causes it for sure. Otherwise feel free to close it.

crssi commented 4 years ago

The @iWARR have go step by step, but we know it isn't TC or ghacksuser.js. I have kept asking to lead him into right direction. 😉

iWARR commented 4 years ago

Good news (inside from the Thorin-Oakenpants) : Comment

Summary: • The Cache API will be recycled into a trash box in FF77. • ServiceWorkers will do the job.

So, I think, it's time to close the issue and keep your fingers crossed. Thank you all, friends, for participation and your kind attention.

Thorin-Oakenpants commented 4 years ago

dude .. the glitch me test has nothing to with appCache

stoically / temporary-containers