search

dexie / Dexie.js

A Minimalistic Wrapper for IndexedDB
https://dexie.org
Apache License 2.0
11.68k stars 642 forks source link

Losing a Webapp's Stored Data Completely on iOS11.3 and iOS11.4 #739

Open StabbarN opened 6 years ago

StabbarN commented 6 years ago

We seem to lose stored data on both localStorage and IndexedDB on iOS 11.3 and 11.4 devices. Upon start of app all data seems to be missing. Our webapp is always installed by first adding it on iOS's home scren, so WKWebView is used. About 30 of 1100 unique users are affected and it seems that some devices lose data frequently (about 10% risk of losing data for every restart of app) and for most users it haven't happened even once. The affected users had a lot of available memory, usually about 22GB of 32GB.

According to http://dexie.org/docs/StorageManager you can control persistance but it seems to not be the case. The function "tryPersistWithoutPromtingUser" stated in http://dexie.org/docs/StorageManager crashes on Chrome and navigator.storage (or navigator.webkitPersistentStorage) is undefined on all iOS versions (tested up to iOS 12 Public Beta 9). According to https://stackoverflow.com/questions/51657388/how-can-i-request-persistent-storage-permissions-in-chrome you can't ask Chrome to query the user to allow persistant storage. Anyway, our problem seems to only occur on iOS.

According to https://stackoverflow.com/questions/15114571/is-localstorage-on-ipad-safari-guaranteed-to-be-persistent we can't assume it's stored until explicitly deleted. At the same time it's irrational for Apple to clear databases on devices with over 22GB available memory (our app only uses at the most a few megabytes).

On iOS 9 and iOS 10 IndexedDBShim (https://github.com/axemclion/IndexedDBShim) is used and it has never been any problem on those versions.

Does anyone else experience these issues? Has Apple changed their "clear web app's cache folder" strategy in recent iOS releases or is there a known bug at some level Dexie/IndexedDB/Safari/iOS?

dfahlander commented 6 years ago

Thanks @StabbarN for reporting this issue.

As you've personally showed me in your logs, the issue happens to so many of your iOS 11.3 and iOS 11.4 users with lots of resources on their machines, and none of your users with earlier versions of iOS. This makes me suspect it could be an issue introduced in iOS 11.3 that occationally wipes out both indexedDB and localStorage. As you showed me, there is nothing in the source code that could do it, and there's no other users than iOS 11.3+ users that has the problem.

Have anyone else experienced this issue? Pinging @beidson. Note: This webapp is installed by adding it on iOS's home screen, so WKWebView is used - if that could have any effect.

Does anybody know a workaround? Would Safari treat the app as more "persistent" if it was utilizing a Service Worker? I've been trying to scan through the version history on technology previews, and it seems the relation between Service Workers and storage has changed in Safari 11.

@StabbarN - Would it be possible to collect the userAgent string from affected devices?

dpogue commented 6 years ago

@StabbarN ~Out of curiosity is your webapp running in Safari, pinned to the homescreen (Web.app), or in its own app using WKWebView or UIWebView?~

EDIT: I should have paid closer attention, you already said it was via pinning to the homescreen:

Our webapp is always installed by first adding it on iOS's home scren, so WKWebView is used.

We haven't (so far) seen any issues with indexedDB in WKWebView on iOS 11.3 or 11.4 in our Cordova apps.

StabbarN commented 6 years ago

@dfahlander User agents: iOS 11.4.1 Mozilla/5.0 (iPad; CPU OS 11_4_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/11.0 Mobile/15E148 Safari/604.1

iOS 11.4 Mozilla/5.0 (iPad; CPU OS 11_4 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/11.0 Mobile/15E148 Safari/604.1

iOS 11.3 Mozilla/5.0 (iPad; CPU OS 11_3 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/11.0 Mobile/15E148 Safari/604.1

The ratio of affected users on 11.4.x and 11.3.x is about 40 to 1 (40 iPads on iOS11.4.x for each 11.3.x).

@dpogue Thanks for sharing that info.

@beidson, as dfahlander also wrote, it would be very interesting to know if Safari has any guidelines how to prevent wipe outs, for example service workers.

Chrome has some guideline here https://developers.google.com/web/updates/2016/06/persistent-storage

Beginning with Chrome 55, Chrome will automatically grant the persistence permission if any of the following are true: The site is bookmarked (and the user has 5 or less bookmarks) The site has high site engagement The site has been added to home screen The site has push notifications enabled

beidson commented 6 years ago

This is a (fairly widely) known issue with save to home screen web apps.

It’s not “a bug in WebKit” but rather a lower level technical issue that we’re tracking internally.

There’s no workaround at this time.

dfahlander commented 6 years ago

Thanks a lot @beidson for sharing this <3

It feels that there must be some practical workaround somehow. For example

What you think @stabbarn? Would any of that be practically doable for you customers?

beidson commented 6 years ago

Of course running in Safari (or a 3rd party WKWebView app) works fine - the issue is only with save to home screen web apps

StabbarN commented 6 years ago

@beidson Thank you very much for your answers! It truly helps us a lot. Does this issue also affect apps using only localStorage and WebSQL? In that case we might start using IndexedDBShim on iOS11/12 as well.

@dfahlander Thank you for your help as well! :) If IndexedDBShim doesn't help then we will probably guide our users to run the app in Safari.

StabbarN commented 6 years ago

Since "There’s no workaround at this time" I assume WebSQL doesn't help, so we will not test that route.

pelme commented 6 years ago

@beidson Is this problem resolved in iOS 12?

dfahlander commented 6 years ago

Would need some advice here. Is there a reported issue somewhere to track? I've really tried to google it with no luck (so far).

@beidson What would be your advice for this customer? Should they wait for a fix (iOS 12 or later) or would the best thing for them be to start the work to convert their app to a appstore app?

beidson commented 6 years ago

I can’t really make business/engineering decisions for anyone, nor can I comment on the timing of any bug fixes in future Apple products/releases.

dfahlander commented 6 years ago

Ok, I understand. Thanks anyway for the response.

pelme commented 6 years ago

For anyone else watching this:

We now have about 15 % iOS 12 users and no one has yet reported this issue and we hope that this means the issue is solved. I think we can close this issue and I will post an update when/if someone of our clients report it for iOS 12.

Since all users running iOS 11 can upgrade to iOS 12, the fix is straightforward. :)

nbrustein commented 1 year ago

I don't know if this is the right place to ask this question or not, but it seems related. I read through the page on How To Use the StorageManager API and did some of my own testing.

It seems like the expectation is that if storage.persist() resolved with true in one session, then you would expect storage.persisted() to resolve with true in a subsequent session.

That is what I am seeing in Chrome and FF, but not on Safari. In Safari,

I'm seeing this in Safari Version 16.3 (18614.4.6.1.6) on OSX and when launching a cordova app in an iOS emulator with iPhone 14 MAX / iOS 16.2)

dfahlander commented 1 year ago

Thanks for sharing your experience. The right place for reporting Safari issues is on webkit bugzilla. A brief search on existing bugs on StorageManager: https://bugs.webkit.org/buglist.cgi?quicksearch=storagemanager