Bug: Toolbox does not handle quote exceeded errors

[tarjei]

If a user has too little space on their harddrive or runs chrome on a partition that is space constrained in other ways, then the serviceworker may start up, but then crash with a cryptic message in the error log:

sw.js:1 Uncaught (in promise) DOMException: Quota exceeded.
Promise rejected (async)
(anonymous) @ sw-toolbox.js:40

IMHO, this is a clear bug. SW-toolbox should at least be able let the client run but with the files uncached. As it is, other sites work, but our site just hangs...

Here's my sw.js:

(global => {
  'use strict'
  // PRECACHE
  // Load the sw-toolbox library.
  global.importScripts('/v3-files/assets/sw-toolbox.js')

  // Turn on debug logging, visible in the Developer Tools' console.
  global.toolbox.options.debug = false

  // Set up a handler for HTTP GET requests for /data/ URLs:

  global.toolbox.router.get(/https:\/\/(localhost|elev.asku.no|gt)(:3000|)\//,
    global.toolbox.networkFirst, {})

  global.toolbox.router.get(/https:\/\/(localhost|elev.asku.no|gt)(:3000|)\/centrifugo.+/,
    global.toolbox.networkFirst, {})

  global.toolbox.router.get(/https:\/\/(localhost|elev.asku.no|gt)(:3000|)\/v3-files\/assets.+/,
    global.toolbox.cacheFirst, {})

  global.toolbox.router.get(/https:\/\/(localhost|elev.asku.no|gt)(:3000|)\/v3-files\/js.+/,
    global.toolbox.networkFirst, {})

  global.toolbox.router.get(/https:\/\/(localhost|elev.asku.no|gt)(:3000|)(media|uploads)\/.+/,
    global.toolbox.cacheFirst, {})

  global.toolbox.router.get(/\/fonts.gstatic.com\/.+/,
    global.toolbox.cacheFirst, {})

  // By default, all requests that don't match our custom handler will use the
  // toolbox.networkFirst cache strategy, and their responses will be stored in
  // the default cache.
  //  global.toolbox.router.default = global.toolbox.networkOnly
  var preCacheFiles = []
  global.toolbox.precache(preCacheFiles)
    // Boilerplate to ensure our service worker takes control of the page as soon
    // as possible.
  global.addEventListener('install', event => event.waitUntil(global.skipWaiting()))
  global.addEventListener('activate', event => event.waitUntil(global.clients.claim()))
})(self)

[rejhgadellaa]

+1. I'm getting Quota Exceeded errors since I updated to Chrome 63 (something changed in it's behaviour?) on a regular basis. It crashes the SW so the whole thing doesn't work anymore (my notifications, among them).

[rejhgadellaa]

Additional: I'm seeing quite a few new "Cache Storage"-related issues on the Chromium bugtracker. This one seems relevant, as SW and CS are behaving weirdly and report an ever growing amount of data stored: https://bugs.chromium.org/p/chromium/issues/detail?id=795134#c9

[gierschv]

+1, could be useful to have a way to handle it, and do a cleaning if we can. It happens a lot on computers with low space, especially chromebooks.

[SadKirK]

+1, it occurs quite rare, but there is no way how to deal with this issue except cache.maxEntries

[tarjei]

We ended up turning off the while serviceworker script bc of this.

[aleluc13]

+1, like @gierschv said, we need a way to handle it. Turning off the SW its not an option for us.

[evelant]

This is affecting me as well. Even if I manually delete all of the caches like so

 caches.keys().then(function(names) {
            for (let name of names)
                caches.delete(name);
        });

The cache usage is not reduced and my SW continues to crash with quota exceeded. The cache is growing with each reload of the page. It grows into the multiple GB before quota exceeded error. I don't understand how deleting all of the caches does not reduce the cache. Maybe a bug in chrome? Regardless sw-toolbox should handle the quota exceeded error.

Edit: It looks like the inflating cache usage was a chrome bug that has been fixed for a future version (looks like v65). https://bugs.chromium.org/p/chromium/issues/detail?id=801024

Regardless, sw-toolbox can still crash with quota exceeded if for example you cache a lot of data in an incognito tab.

[jeffposnick]

Hello folks—Thanks for your patience.

There were a few Chrome-related bugs, along with some changes in policy starting with Chrome 63, that led to an uptick of these types of failures. But it's definitely something that could crop up, given enough cache storage usage, on any browser.

I've made some changes in a branch focused on trying to avoid scenarios in which you end up not getting a valid Response returned to your page.

I'd be very grateful if folks who can consistently reproduce these failures try

importScripts('https://rawgit.com/GoogleChromeLabs/sw-toolbox/catch-quota-errors/sw-toolbox.js');

to pull in the updated code, and let us know how it behaves. You should see a friendly error message logged when a cache.put() fails, and it shouldn't prevent the Response that we already have from being returned to the page.

I think that's the extend of the change we could make at this point. sw-toolbox has provisions for opting-in to automatic cache expiration, so I don't think it makes sense for its code to forcibly delete entries beyond those that the developers have asked it to manage.

[evelant]

The way the chrome bug affected me and what I am concerned about solving is the case where users end up "stuck" with an old version of the application. This was happening to me (using sw-precache-webpack-plugin) when the quota was exceeded. The SW would fail to cache the new assets and that left the user stuck with the old cached version essentially permanently. This situation is really bad because users are stuck with old app versions which may not function correctly anymore and there is no way for them to know what happened or how to resolve it.

[rejhgadellaa]

Btw, I think a similar issue is present in workbox-sw. I haven't been able to check and make sure because I have no data usb-c cable I can plug into my phone and desktop has 19gb quota so that's taking a while to hit the max. I have experienced issues with wb-sw on android twice now, though, and clearing site data for the domain fixed it (just like it does for sw-toolbox).

Should I create an issue on the wb-sw tracker with this limited info or..?

[jeffposnick]

@rejhgadellaa, there's an existing issue on https://github.com/GoogleChrome/workbox/issues/1308 to track the Workbox story. Feel free to chime in there.

@AndrewMorsillo, thanks for filing https://github.com/GoogleChromeLabs/sw-precache/issues/345 to talk about the sw-precache side of things. I'll follow up there, as solving this for assets added in the install handler will probably require a different approach than what we can do for assets that are cached at runtime.

For anyone looking for ways to trigger quota issues, you should know that a Chrome Incognito window is capped at ~120mb of total storage quota (regardless of how much space is available on your device), and that storing an opaque response will take up ~7mb of quota.

I'd really love to hear from folks who have encountered quota issues with assets cached at runtime using sw-toolbox whether the changes in that branch are a step in the right direction.

[hopetambala]

@AndrewMorsillo did your problem still persist?

importScripts('https://rawgit.com/GoogleChromeLabs/sw-toolbox/catch-quota-errors/sw-toolbox.js'); worked perfectly for me. Is this ok to use on a production level?

[tarjei]

@jeffposnick I ended up migrating to workbox and changed some parameters to reduce the overall storage size. T