Using Dexie with Redux

[varghesep]

I'm looking at this AngularJS Dexie Redux example.

1. Is the pattern used here is to first update the IndexedDB and then the redux store when a new todo is added?
2. What if there are hundreds of todos (200) and the redux store having all of them which can impact the performance? If the app can display only 10 todos out of 200, should the redux store have only 10 and not 200?

[nponiros]

1. Yes I first update the db and then the store similar to how I would first send data to the server before updating the store with the new todo (pessimistic update). But you could also first update the store and then the db and in case the db update fails remove the todo from the store again (optimistic update).
2. This is up to you and depends on your requirements. There isn't one definitive answer I can give here. There are various trade-offs to consider.

[varghesep]

I think the best practice in every situation would be to update the persistent store (IndexedDB or SQL) and retrieve whatever necessary for the redux store based on the app's query status.

[nponiros]

Well you can also preload data which you think the user might need next. Reading the store is faster than reading the persistant storage. But as I said it's up to you. I use multiple strategies depending on the requirements.

[varghesep]

My situation is described in this ticket.

[nponiros]

Even reading that ticket, I personaly cannot give a definitiv answer. Why do you even want to use dexie?

Does your app have to be able to work offline? If yes you probably would want all the contacts in indexeddb (assuming that 5000 contacts don't reach the limits of the db).

If you want to use dexie to speedup subsequent queries then I would leave all data on the server and cache each query in indexeddb so I can use that for subsequent calls using the same query. In case the server data changes you would need some way to invalidate that cache.

You could also do what you described and first query the local db and then the server db. In case the server data changes you would always have to query both dbs.

Each strategy has different trade-offs and keep in mind the user can remove all data from the local db.

[varghesep]

The reasons are (1) Dexie's bulk update feature to cache the data locally fast (2) to make the app work offline (3) save money on cloud access.

In my app, the data changes significantly every day in the first few weeks, then after a few weeks of use, the data becomes almost static and may change only once every week or so.

[nponiros]

Okay I think I got a better picture now of what you need so here are some more ideas/scenarios. I wouldn't consider any of this best practice and I don't think there really are best practices for the questions you are asking because the implementation really depends on details which I currently don't have.

First off we should differentiate the various meanings of "offline". I will only consider webapps and not native or hybrid apps which could for example be installed via google play or apple's app store.

1. User is online and loads the webapp and the data but goes offline for some time because suddenly there is no network connection or has a flaky/slow connection.
2. User is online and loads the webapp and the data and then goes offline on purpose by for example disabling the WiFi and you want the user to be able to continue working. I assume here that the user will remain offline for hours or even days.
3. User is offline and wants to use the webapp (Assuming that at some point the webapp/data was loaded whilst the user was online). I assume here that the user will remain offline for hours or even days.

For 1. you could leave the data on the server and only cache data which the user already explicitly loaded. So basically approach 2. from https://github.com/dfahlander/Dexie.js/issues/552 . The user will be able to work with the already loaded data and in case more data is needed the user would have to wait for a better connection. Additions and changes can be stored locally and sent to the server once the user is again online. In this case you would also need some way of invalidating the cache. Lets say query 1 returns 10 contacts today but 3 days from now that same query might return 12 contacts because there is newer data on the server. How and when to do cache invalidation depends on how important it is for the user to have the latest data. Maybe you would want to invalidate after a few hours in the first weeks and after a few days later. You could also invalidate the cache directly in case the user is online.

For 2. you would most probably want to have as much data as possible in indexeddb so that the user can work fully offline after the initial load. So basically approach 1. from https://github.com/dfahlander/Dexie.js/issues/552 . You don't have to load all data at once. You could load lets say the first 100 contacts during initial load and load the rest in the background until you got all data (assuming that the user will remain online long enough). In this case I would not treat the IndexedDB as a cache but as a second database which I would want to synchronize with the server database. So you would probably want to implement some syncing algorithm to make sure that you only get new data from the server and not all data belonging to a query. In this scenario every query/addition/update will be only executed locally with the implicit assumption that some data might be old. Dexie-syncable might help you with the synchronization, but you would have to implement the synchronization algorithm on the server. Depending on who can change the data (only one person can change the data, multiple people can change the same data etc.) you will also have to manage conflicts in case 2 people change the same dataset.

For 3. you would need all of 2. + a way to have the complete webapp cached in the browser. For that you can look into AppCache and/or Service workers depending on the browsers you need to support.

Now regarding your original question about redux store (part 2). I wouldn't want 5000 contacts in the store all the time and I also would not want to display 5000 contacts all the time. I would implement some sort of pagination for the contacts list and keep in the store the contacts which are currently displayed. For an added performance I might also keep contacts that the user already saw in the store and users that I assume the user will want to see next. Whether I would do this depends on how pagination is implemented and whether I can be relatively sure to know what contacts will be needed next.

For part 1: as I already said it depends on whether you want optimistic or pessimistic updates. I usually use pessimistic updates because those are easier to implement. Since updating IndexedDB is relatively fast compared to sending the data to the server, I would stick with pessimistic updates in your case.

So this became a lot longer than expected. I hope it actually helps you with your implementation.