Unable to upload more than 10GB of Local Backgrounds (Firefox)

[DSHowarth]

Issue: When adding a background or group of backgrounds that would take the sum total of backgrounds uploaded over 10GB, the extension hangs and I see this error:

Why this matters: it probably doesn't to anyone besides me. I was uploading webp animated backgrounds as the extension very easily handles displaying them, but their size can stack up quickly.

Further digging seems to imply that this is a limit imposed by Firefox:

--

In Firefox, the maximum storage space an origin can use in best-effort mode is whichever is the smaller of:

10% of the total disk size where the profile of the user is stored. Or 10 GiB, which is the group limit that Firefox applies to all origins that are part of the same eTLD+1 domain.

Origins for which persistent storage has been granted can store up to 50% of the total disk size, capped at 8 TiB, and are not subject to the eTLD+1 group limit.

For example, if the device has a 500 GiB hard drive, Firefox will allow an origin to store up to:

In best-effort mode: 10 GiB of data, which is the eTLD+1 group limit. In persistent mode: 250 GiB, which is 50% of the total disk size.

Note that it might not actually be possible for the origin to reach its quota because it is calculated based on the hard drive total size, not the currently available disk space. This is done for security reasons, to avoid fingerprinting.

--

Bonjourr and Firefox are installed on a hard drive with 800gb of free space, so that is not where the limit is coming from.

Things already tried: Editing manifest.json to include 'unlimitedStorage' as a permission Adjusting values for dom.storage.default_quota

I'm not an addon developer, so I'm not sure whether it's something that can be adjusted either on the developer side or client side. But I'll happily continue using the extension either way. Thanks!

Firefox v 132.0.2 64bit Bonjourr v 20.2.2

[victrme]

Thanks for the research @DSHowarth !

unlimitedStorage does not do anything since we are now using indexedDB to store backgrounds. I think that pasting:

navigator.storage.persist()

and allowing the persistant storage will switch Bonjourr from "best-effort mode" to "persistent mode", increasing the limit to 50% of the disk size.

I didn't know about storage.persist and your use case is very interesting, we'll look more into it when we'll work on the background update!

Also, talking about lots of backgrounds to add, #424 is somewhat related.

[DSHowarth]

navigator.storage.persist() seemed to work a treat. I threw it at the top of main.js in my Firefox release folder:

And Firefox seems to be happy to let me exceed the 10Gb limit:

This is definitely the laziest implementation of persistent storage. So until this gets added to the main extension, I'll run my unlisted version and report back if I'm running into any bugs from this.

Thanks, and hope this helps your roadmap.

[Gitoffthelawn]

This is definitely the laziest implementation of persistent storage. So until this gets added to the main extension, I'll run my unlisted version and report back if I'm running into any bugs from this.

As you've seen, web browsers these days have all sorts of different storage types (it can get confusing at times). Are there any downsides to using this type of storage? I'm largely thinking: performance, privacy, compression (disk space), and storage efficiency (e.g. automatic cleanup and freeing of storage when items are removed).

[victrme]

• performance: IndexedDB is the fastest browser storage for storing huge amount of data. This is the reason why I took the time to migrate local backgrounds from storage.local to indexedDB!

• privacy: As private as your browser can be, so not the best. It is still better in my opinion than Bonjourr having permission to access your device's files.

• disk space: Not good, because it has to duplicate the backgrounds you add to Bonjourr. Although the files are saved as ArrayBuffers instead of dataURI, so still better than storage.local

• efficiency: With navigator.storage.persist, no cleanup! It seems like the data will never be automatically removed by the browser.