dataloss / offline cache showing while online

[iturn]

Sometimes the database seems to load from offline cache while its online. The data is written to the database and received in firebase and reflecting on other clients. But the current client continues to show the outdated data, a hard refresh in chrome seems to fix this. Any idea?

[iturn]

I fixed the issue via a workaround, probably the issue isnt in angularfire2-offline

[iturn]

Issue still persists , maybe its an issue of this package

[iturn]

After much debugging the cause data loss was the following. On slow computers the offfline write takes time to execute. Sometimes a lot like 10 seconds, if a user runs another similar write the promise gets broken somehow. A callback when offline write comples would be awseome. For example like this, although not sure this is the correct place to do this in the package.

//offline-write.js

export function OfflineWrite(firebasePromise, type, ref, method, args, localUpdateService) { localUpdateService.update('write', function (writeCache) { if (!writeCache) { writeCache = { lastId: 0, cache: {} }; } writeCache.lastId++; writeCache.cache[writeCache.lastId] = { type: type, ref: ref, method: method, args: args }; return writeCache; }).then(function (writeCache) { var id = writeCache.lastId; firebasePromise.then(function () { WriteComplete(id, localUpdateService);

    }).then(console.log("LocalStorageServer:: offline write done"));
});

}

The cause and fix of offline database showing while offline and not syncing is not yet clear

[adriancarriger]

Hi @iturn thanks for posting this issue! The easiest way for me to fix this is if I can reproduce it, so I need a little bit more info:

1. Is the app online the entire time?
2. Is the similar write made to the exact same reference?
3. Are you experiencing the issue with a list or object?
4. What kind of write are you making (push, set, etc)?
5. Can you show your code, or show a minimal example that reproduces the issue?

[iturn]

Thanks for your awesome work on this repo and your response! Im here for any questions or additional info.

All of the below is related to the slow write on OfflineWrite while CPU is high, or a slow/old computer is used. I use a single core VM with CPU load app to simulate this, but even on this VM with cpu stress its much faster then on a new system with Intel Celeron N3060 of which my client bought 15 :-(. On these slow computers the offline write after calling sometimes takes up to 15 seconds while on my vm approx 3. I would also like to mention that all of the above is a complete non issue on decent hardware, since the offline write completes fast.

1. The app is loaded online, then internet is disconnected
2. I don't think the writes are to the exact same reference. But to clarify and example and saying this, they are made to children of the same node. /1234/1234/1234/a : "12" /1234/1234/1234/b : "13" /1234/1234/5678/a : "14" /1234/1234/5678/b : "15"

3-4. its and update object call

5. See the img for the console logs called and the time delay between the 2 on a fast computer. the object written in the update object call is very simple 4 string properties on a database object with around 15 mixed properties.

// function in some-service updatePerson(key: string, data):void { this.angularFireOfflineDatabase.object(/projectPersons/${this.personsKey}/${personKey}) .update(data ) .then(console.log("ProjectDbService:: receveived person update")) }

// offline-write.js in angularfire2-offline package export function OfflineWrite(firebasePromise, type, ref, method, args, localUpdateService) { localUpdateService.update('write', function (writeCache) { if (!writeCache) { writeCache = { lastId: 0, cache: {} }; } writeCache.lastId++; writeCache.cache[writeCache.lastId] = { type: type, ref: ref, method: method, args: args }; return writeCache; }).then(function (writeCache) { var id = writeCache.lastId; firebasePromise.then(function () { WriteComplete(id, localUpdateService);

    }).then(console.log("LocalStorageServer:: offline write done"));
});

}

[adriancarriger]

Thanks for the info @iturn ..I'll be working on reproducing the issue 👍

[iturn]

If any help needed with reprodction, please advice.

[adriancarriger]

Hi @iturn ..this is a tricky issue to reproduce so I'll just walk you through what I've done so far:

I've attempted to reproduce the issue here (live demo).

It runs two queries on sibling references quickly in hope that the second will overwrite the first in this way:

console.log('starting write #1');
this.objectExample.update({maxSpeed: 100});
setTimeout(() => {
  console.log('starting write #2');
  this.objectExample.update({engine: null});
}, 3);

Steps Taken

Please try these steps to tell me if you get the same results:

1. Visit the demo

2. Press the Reset example (press while online) button

3. Open chrome dev tools, go to Application, then Clear storage, and click Clear site data

4. In dev tools go to Performance, then under CPU: set the dropdown to 20x slowdown

5. Refresh page

6. Go to dev tools Application, then IndexedDB

   • This part is weird for me.. sometimes I have to refresh to page for the IndexedDB part of Chrome to show anything, and sometimes I have to restart Chrome itself. If you don't see what is in the following image, please refresh or restart Chrome, and it should show.

7. Drill down to localforage then keyvaluepairs and expand the value related to the key read/objectcar. Your screen should look like this:

8. Disconnect from the internet and go to the dev tools console

9. Press Simulate two writes (press while offline)

10. In the dev tools console you should see this:

11. Go to dev tools Application, then IndexedDB and expand all the values. You should see this:

12. Go online

13. Go the the console, and you should see this:

Results

Please let me know if you're seeing the same thing as me. In my case here's what's happening

• The IndexedDB value for "write" has an object with a key called cache. It contains all of the writes that are "replayed" when the user goes back online. They are all saved separately. When you make multiple offline writes, a new write should not overwrite a previous write even if it is still getting saved.

• In the demo I've added logging for when an offline write completes. It should also log the internal unique id which will line up with the cache key inside the IndexedDB value for "write".

• If you see two completion messages with two different internal ids, then you know that your offline writes are probably working correctly.

• Each message will look something like this Finished offline write (internal id: 1)

Next steps

If you you get a message for both writes then you must be losing data during something that occurs after the offline write is complete. Let me know what happens. Thanks!

[iturn]

Indeed you are right, i have exactly the same result as you do following your steps.

Im not sure where to continue debugging this issue, if the conclusion is that the issue is in my application and not in the package

[iturn]

although i now activated the cpu loader app and again see this delay of 15 seconds

[adriancarriger]

I hope we're getting closer to solving this! I have a few questions:

• What do you mean by "CPU loader app"?
• What do you mean by "100%"? Is that the CPU throttling? In Chrome 59 it shows me options like 20x slowdown
• Did you click the button once and get the duplicate messages?
• Can you list all the actions you took step-by-step (especially what you did differently the second time)?

[iturn]

This is an example of a CPU loader app for windows; https://www.jam-software.com/heavyload/ I dont have much insights in the chrome cpu options. So i applied both 20X slow and the CPU loader app. I did click the button once without cpu loader, and 1 time with. the steps are exactly like you described, just 1 variable, the cpu loader app.

[adriancarriger]

So what I understand is that while using the CPU loader app it takes 15 seconds to save. When do you experience the dataloss?

[iturn]

When the application is closed within that 15 seconds period. Or when another call is made and the still pending call gets overwritten/deleted.

Kind Regards, Met vriendelijke groeten

Stijn Post

+31 6 38 75 46 44 www.iturn.it stijn@iturn.it

On 22 June 2017 at 14:33:21, Adrian Carriger (notifications@github.com) wrote:

So what I understand is that while using the CPU loader app it takes 15 seconds to save. When do you experience the dataloss?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/adriancarriger/angularfire2-offline/issues/44#issuecomment-310366548, or mute the thread https://github.com/notifications/unsubscribe-auth/ALurUqkvNtBFawBg5Qgo-rRjxkGMrH0qks5sGl8QgaJpZM4N3NZJ .

[adriancarriger]

I now understand! You would like to warn the user if they are closing the app before the save is complete.

A possible solution would be to add an additional promise for the offline save that you could use like this:

const promise = this.afoDatabase.object('car').update({maxSpeed: 100});

promise.offline.then(() => console.log('offline write complete!'));

Let me know if that would work. Thanks!

[iturn]

Yes you are right,. that would fit perfectly!

[adriancarriger]

Offline write promises now work in version 4.2.1! 🎉

Demo make sure to look at the console (with Code)

Usage

const promise = this.afoDatabase.object('car').update({maxSpeed: 100});
promise.offline.then(() => console.log('offline write complete!'));

[iturn]

Great news! Thanks :)