search

rt2zz / redux-persist

persist and rehydrate a redux store
MIT License
12.95k stars 866 forks source link

Data sporadically erased (React Native) #809

Open jln-dk opened 6 years ago

jln-dk commented 6 years ago

Introduction

We are building a React Native app, and we are using redux-persist together with redux-persist-filesystem-storage to store data on the phone.

The issue

We are experiencing that sometimes after an app update (via TestFlight and Play Store) all the app data just is erased. The app is simply back to initial state. This can happen for both iOS and Android.

Steps to reproduce

Well, this is where it get's annoying.. We haven't yet been able to get the exact steps to reproduce the issue. It just happens sporadically. We have around 100 beta testers, and ~20 people have reported this issue. Both on iOS and Android, and there was no link between a specific app update version or anything.

Similar issues

We have of course looked around on GitHub, trying to find similar issues. At first we thought https://github.com/rt2zz/redux-persist/issues/199 was our issue, so that's why we switched to redux-persist-filesystem-storage from AsyncStorage. But that didn't help. And it looks like other people are still experiencing it as well: https://github.com/robwalkerco/redux-persist-filesystem-storage/issues/2

Additional information

Final note

Although this issue is somewhat vaguely described, I'm submitting it to raise awareness, because we think it's a rather important issue if it really IS an existing bug. Maybe someone out there is looking for help with the same issue - so here it is.

I have created the same issue on redux-persist-filesystem-storage: https://github.com/robwalkerco/redux-persist-filesystem-storage/issues/14

ghost commented 6 years ago

+1 !

alexmngn commented 6 years ago

We're actually having the exact same problem in my company. We are just using AsyncStorage though. @jesperlndk did you find a solution? Or a different way to store the data?

I've also noticed sometimes we get the following error:


{
  err: Error: redux-persist: persist timed out for persist key "myKey" at blob:http://localhost:8081/d0182199-b2a0-4806-ad16-47af460661d8:18512:45,
  key: "myKey"
  payload: undefined
  type: "persist/REHYDRATE"
}
jrmurad commented 6 years ago

Same here. I started looking into it and I think what's happening is the getItem() Promise on app load rejects with an error. That error is passed to the REHYDRATE action. The action does not consider the error and just writes undefined to the store, causing the setItem middleware to run (because it detected a "change") and "erasing" the data.

I am currently looking for issues with a suggested fix or a PR. Might make my own PR but I'm not sure what correct behavior should be. Probably just to not write anything to local store if getItem failed. And I might need to add a new error handler to the config so consumer can handle when a rehydration error occurs.

alanlanglois commented 6 years ago

@jrmurad I moved from AsyncStorage to FSStorage (based on react-native-fs) and the issue seem to appear way less often, but it still occurs from time to time on some Android devices. An appropriate fix would be to have a way to detect and display they error and stop the rehydrate process.

lilosir commented 6 years ago

Same issue, any updates?

lukebrandonfarrell commented 6 years ago

I can confirm that this still happens when using FSStorage. It happens of both v5 & v4. Seems to have something to do with the autoRehydrate which is not getting logged and data not persisted in some cases.

lukebrandonfarrell commented 6 years ago

After more testing it seems to be working with redux-persist-filesystem-storage without erasing data randomly. It could be something to do with AsyncStorage. @lilosir

alanlanglois commented 6 years ago

Anyone found a solution @jrmurad @jesperlndk @alexmngn?

lukebrandonfarrell commented 6 years ago

@alanlanglois Using redux-persist-filesystem-storage worked for me. Using in production.

alanlanglois commented 6 years ago

@lukebrandonfarrell Few mounths ago I gave it a try, unfortunatly it wasn't working with an expo detached project. (rn-fetch-blob wasn't at least). I then changed a little the lib to use react-native-fs instead of rn-fetch-blob. It fixed the problem in most cases, but I still have the issue from time to time (way less often). Do you have a large audience are you sure it totally fixed your problem? (I got like 10K+ install on Android)

ssorallen commented 6 years ago

I was able to reproduce the issue by setting timeout to something incredibly low like 10ms. The store was consistently overwritten by initialState. That's when I tested a few weeks ago, and I didn't write a test. I'd like to write a test to reproduce it, but I wanted to mention that in case anyone else is able to investigate further.

timeout is a key in the persist config, but it is not yet documented in the README.

alanlanglois commented 6 years ago

@ssorallen It seems to be talked here: https://github.com/rt2zz/redux-persist/issues/717

ssorallen commented 5 years ago

redux-persist's timeout functionality definitely wipes out the store and rehydrates it with initial state. I don't know if that's the only issue here, but I have confirmed locally and in my own app that it does wipe out my store.

You can disable the timeout functionality entirely by passing timeout: 0 in your persistConfig. If your store takes a while to rehydrate then the PersistGate will continue to show the loading state. If your store never rehydrates then the PersistGate will never go away. However, it won't ever hit the timeout functionality and wipe your store. Your users can refresh the app or whatever it takes in case that slow rehydration was transient.

In your persist config, prevent the timeout setup (persistReducer.js#L88-99) (the default timeout value is 5000):

const persistConfig: {
   ...
   timeout: 0, // The code base checks for falsy, so 0 disables
};

A complete solution for redux-persist would be to add a timeout component to PersistGate that would render if your timeout is hit and stop the rehydration work. What it shouldn't do is then rehydrate your store with initialState, which is the current functionality.

wmonecke commented 5 years ago

Happening on my app as well! Some of my users have reported to me that all their journal data disappeared. Have not been able to reproduce.

"react": "16.8.3",
"react-native": "0.59.8",
"react-redux": "^6.0.0", 
"redux": "^4.0.1",
"redux-persist": "^5.10.0",
"redux-thunk": "^2.3.0",
alxmrtnz commented 5 years ago

I've been working on an Expo-based React Native app that encountered the same problem.

After trying the timeout fix mentioned above and finding that didn't work for us, we ended up using redux-persist-expo-fs-storage and haven't been experiencing the same problems since.

If you're using Expo, I'd recommend trying that out as your storage solution. Just for reference, here's some more package.json info:

    "react": "16.5.0",
    "react-native": "https://github.com/expo/react-native/archive/sdk-32.0.0.tar.gz",
    "react-redux": "^5.1.0",
    "redux": "^4.0.1",
    "redux-persist": "^5.10.0",
    "redux-persist-expo-fs-storage": "^1.2.2",
    "redux-thunk": "^2.3.0",
webytecno commented 4 years ago

This bug is also happening in our production app. We have a lot of users complaining because their data (stored using the library: redux-persist-fs-storage), is being sporadically erased... We tried adjusting the timeout config, but that didn't solve the problem. We are not using Expo. If anyone solved this issue, please help me with the solution. Many thanks!!!!

svenlombaert commented 4 years ago

We're experiencing the same issue as well now (for a couple of months). I can give this information (which is about the same as the OP)

If there's any way we can help out (by providing logs, setups, etc) please contact me. It's becoming a big problem for us and our users.

ssorallen commented 4 years ago

Is this repo maintained any more? These issues date back a full two years now, and the last commit to master was September 2019.

If this is affecting your production application and causing data loss, it might be time to find a replacement for redux-persist.

svenlombaert commented 4 years ago

Well, I think why people are still posting all the issues here is because when you're using redux, there doesn't seem to be a lot of alternative to redux-persist.

Orange9000 commented 4 years ago

In my case explicitly setting the whitelisted reducers, as well as timeout to 0 in redux persist config actually helped. No more store data loss on app restart after that.

svenlombaert commented 4 years ago

I think by now, I've tried every possible combination of workarounds. They always get persisted and rehydrated except on app update. We implemented our own solution now to persist a specific reducer which contained data that never gets synced with the server.

lukebrandonfarrell commented 4 years ago

Btw this issue drove us to build our own persist library (a year ago): https://github.com/aspect-apps/redux-persist-machine

As we got frustrated with little issues like this, we wanted a much simpler implementation, with more customisation over the data we load, and how we load it.

wmonecke commented 3 years ago

Does anyone have an update on this? This is actually making us lose customers.

ArchanaNair commented 3 years ago

Any fix on this issue?

ssorallen commented 3 years ago

@wmonecke @ArchanaNair If redux-persist is making you lose customers, I’d likely abandon this library. It has not been maintained in several years at this point. (see dates on issues and comments)

You might have more success with major version 5, but if these are decisions that impact your revenue then this library does not seem reliable enough.

StuartGough7 commented 3 years ago

I managed to identify and solve the same issue on Android using redux persist with async storage. Here is what the problem was in my case: As @jrmurad mentioned the error manifests itself on getItem. To identify it I patched the persistReducer.js as follows and threw an error instead of rehydrating with undefined:

diff --git a/node_modules/redux-persist/lib/persistReducer.js b/node_modules/redux-persist/lib/persistReducer.js
index 1116881..6aed0e7 100644
--- a/node_modules/redux-persist/lib/persistReducer.js
+++ b/node_modules/redux-persist/lib/persistReducer.js
@@ -94,6 +94,7 @@ function persistReducer(config, baseReducer) {
       if (typeof action.rehydrate !== 'function' || typeof action.register !== 'function') throw new Error('redux-persist: either rehydrate or register is not a function on the PERSIST action. This can happen if the action is being replayed. This is an unexplored use case, please open an issue and we will figure out a resolution.');
       action.register(config.key);
       getStoredState(config).then(function (restoredState) {
+        console.log('redux persist reducer')
         var migrate = config.migrate || function (s, v) {
           return Promise.resolve(s);
         };
@@ -102,11 +103,14 @@ function persistReducer(config, baseReducer) {
           _rehydrate(migratedState);
         }, function (migrateErr) {
           if (process.env.NODE_ENV !== 'production' && migrateErr) console.error('redux-persist: migration error', migrateErr);
-
-          _rehydrate(undefined, migrateErr);
+          console.log('redux persist migration error', migrateErr)
+          throw migrateErr
+          // _rehydrate(undefined, migrateErr);
         });
       }, function (err) {
-        _rehydrate(undefined, err);
+        console.log('redux persist error', err)
+        throw err
+        // _rehydrate(undefined, err);
       });
       return _objectSpread({}, baseReducer(restState, action), {
         _persist: {

In my case this yielded an error sometime later (due to this error being not very repeatable) as follows:

redux persist error [Error: Row too big to fit into CursorWindow requiredPos=0, totalRows=1] This lead me to the local SQLite issue of trying to read big text entries for single rows exceeding the default of 4mb for a single row, and that indeed was the issue. A few thousand rows of text were being stored in the redux store under a single row.

Saving this as a JSON file and storing a reference to that file in redux solved the issue. You could expand that limit but I figured it's probably there for good reason. I hope this saves someone else.

Shared credit to @Federkun

wmonecke commented 3 years ago

@ssorallen I have been searching for a replacement with no success!

Shazil1 commented 3 years ago

I am getting this issue where i store the userType in the redux-persist but when ever the app is updated it loses all its data and the app becomes un useable the user have to log out in order to store the userType from the api i have been looking for hours but still was'nt able to find a solution

r1jsheth commented 3 years ago

We are still facing this, seems like this repo is not actively maintained

r1jsheth commented 3 years ago

I think by now, I've tried every possible combination of workarounds. They always get persisted and rehydrated except on app update. We implemented our own solution now to persist a specific reducer which contained data that never gets synced with the server.

Would you mind sharing what did you do?

Orange9000 commented 2 years ago

I am getting this issue where i store the userType in the redux-persist but when ever the app is updated it loses all its data and the app becomes un useable the user have to log out in order to store the userType from the api i have been looking for hours but still was'nt able to find a solution

What platform does that happen on? We too are facing the same issue on iOS, but that doesn't happen 100% of time (which compicates things) We're using redux-persist-filesystem-storage, thus it could be related to #https://github.com/robwalkerco/redux-persist-filesystem-storage/issues/44 Fixing this is rather hard since its almost impossible to reproduce due its random nature.

Orange9000 commented 2 years ago

For anyone experiencing data loss after app update on iOS, this is probably not the redux-persist issue, but of a storage engine, or its filesystem interaction logic, to be exact.

That my happen if a storage engine keeps persisted state in a document directory, path to which may look something like this data/Containers/Data/Application/APP_UUID/Documents/, where APP_UUID is a unique identifier which changes on app updates. I suggest, after an app update, storage engine, for some reason, tries to look for persisted state in an old directory, finds nothing, which in return causes redux-persist to revert the store to its default state.

Why does that happen is a mystery, since this error is not 100% reproducible.

In the end, that is just my suggestion, which may not be the case.

haozhutw commented 2 years ago

For anyone experiencing data loss after app update on iOS, this is probably not the redux-persist issue, but of a storage engine, or its filesystem interaction logic, to be exact.

That my happen if a storage engine keeps persisted state in a document directory, path to which may look something like this data/Containers/Data/Application/APP_UUID/Documents/, where APP_UUID is a unique identifier which changes on app updates. I suggest, after an app update, storage engine, for some reason, tries to look for persisted state in an old directory, finds nothing, which in return causes redux-persist to revert the store to its default state.

Why does that happen is a mystery, since this error is not 100% reproducible.

In the end, that is just my suggestion, which may not be the case.

You are right, the APP_UUID may be changed after app update, but shouldn't the store engine save and query data from a relative path? APP_UUID may be changed, but the Document folder is always there. This should not be the case since same issue also happens on Android. This issue happens on production apps and it's still not resolved after 3 years. Looks like this library is not maintained, we need to try a replacement even though it's tough.

Ti-tanium commented 2 years ago

We are facing the same issue, and I think I find a way to reliably replicate this bug. On Android 9 emulator, almost every time I restart the app, the data is erased.

wanted-o commented 1 year ago

Hello, we had the same problem on production, data randomly erased on IOS. How we can resolve the issue? Someone has solution?

kangfenmao commented 1 year ago

fixed by https://github.com/rt2zz/redux-persist/issues/199#issuecomment-1473247158

jeroenheijmans commented 1 year ago

I'm also in the progress of debugging a possible variant of this issue. I wanted to share what I have so far, and might edit to update this comment in the near future if I find out more. The solution you may want to implement depends on your application needs I suppose. I don't have a good solution yet for my own actual current production scenario, partially because I'm unsure yet which variant I'm (mostly) experiencing.

Variant 1: misbehaving transforms (confirmed in production for me)

If I introduce an exception in the transforms that are applied after reading from storage, things will blow up and store data might get reset to initial state. In pseudo-code (I don't have a clear repro at hand yet, have to extract that from our prod app):

const persistConfig = {
  key: 'root',
  storage: YourStorageHere,
  transforms: createTransform(
    (inboundState, key) => ({ ...inboundState, myRandomValue: Math.trunc(Math.random() * 10) }),
    (outboundState, key) => {
        if (myRandomValue[0] === 5) throw "simulated random error";
        return { ...outboundState };
    },
   { }
};

So every 1 in 10 times storage will have myRandomValue === 5 and when it is read from storage the tranform will crash. This simulates for example a more realistic NullReferenceException in the outbound function. And it will cause the state to be reset to initial state for your store.

Variant 2a: storage timeouts (confirmed in snack.expo.dev)

Earlier comments, most notably the one by @ssorallen already mention it might have to do with timeouts. I can in fact confirm that this is the case! I wrote this storage implementation and used it on snack.expo.dev to see the behavior in action. First, the code:

const getDelay = () => Math.trunc(Math.random() * 10000);

function log(...args) {
  console.log(`[${new Date().toISOString().substring(11)}]`, ...args);
}

const persistConfig = {
  key: 'root',
  storage: {
    getItem(key) {
      const delay = getDelay();
      log(`[Delayed ${delay.toString().padStart(6, ' ')}] Getting`, key);
      return new Promise(resolve => {
        const raw = localStorage.getItem(key);
        log(`[Delayed ${delay.toString().padStart(6, ' ')}] Getting resolved to`, raw);
        return setTimeout(() => {
          log(`[Delayed ${delay.toString().padStart(6, ' ')}] Getting resolves now`);
          resolve(raw);
        }, delay); // ⛔ !!! 50/50 chance this goes above the 5000ms timeout!
      });
    },
    setItem(key, value) {
      log(`[Delayed ${delay.toString().padStart(6, ' ')}] Setting              `, key, value);
      return new Promise(resolve => {
        log(`[Delayed ${delay.toString().padStart(6, ' ')}] Setting resolved     `, key, value);
        return setTimeout(() => {
          log(`[Delayed ${delay.toString().padStart(6, ' ')}] Setting resolves now `, key, value);
          localStorage.setItem(key, value);
          resolve();
        }, 50); // Small delay
      });
    },
    removeItem(key) {
      log('Removing', key);
      return new Promise(resolve => {
        log('Removing resolved', key);
        return setTimeout(() => {
          log('Removing resolves now', key);
          localStorage.removeItem(key);
          resolve();
        }, getDelay()); // Random delay
      })
    }
  },
};

This storage implementation retrieves the state with a random delay of 0-10 seconds. If it exceeds the default redux-persist timeout of 5 seconds, state will get reset. So it is a roughly 50/50 chance of getting wiped.

You can check my 'Snack' example live to see this in action. Increment the counter a few times to persist that state. Then turn preview off and on again to "reload" the app. The state gets read from storage, and you have 50/50 chance of it being either loaded, or reset to initial state.

Variant 2b: storage errors (confirmable in previous snack.expo.dev)

If you change the getItem(...) method for the storage implementation to this:

getItem(key) {
  const delay = Math.trunc(Math.random() * 1000);
  return new Promise((resolve, reject) => {
    const raw = localStorage.getItem(key);

    if (delay % 2 === 0) {
      console.log('🛑🛑🛑 Simulating error!');
      return setTimeout(() => reject('error simulated'), delay);
    }

    return setTimeout(() => { resolve(raw); }, delay);
  });
},

It will randomly fail about 50% of the time (in this example getItem always resolves within the default timeout of 5000. And that also has the effect of resetting to initial state.

Depending on your storage implementation (we use react-native-encrypted-storage in our production app) there might be scenarios where this happens. Who knows, Android or iOS might have all sorts of reasons to throw a (probably temporary) error while accessing storage. For example the SD card might not be ready for reading, would (I imagine) not be an uncommon scenario.

Variant 3: Storage being 'full' (unconfirmed for me for the moment)

Just to make my comment a bit more "complete" I've added this variant. The Original Post of this issue already mentions and links a related issue that suggests the same behavior might be caused by a filled up storage causing a reset. If you're reading this, you might want to consider if this is your issue. For my production scenario it's an unlikely cause, as our state is fairly small in size.

PS. This might be different from variant "2B" above because it would happen during setItem, but in essence it would also be a bit of the same: errors may occur during storage operations.