OK, back to "localStorage"

[steida]

This was a long journey looking for the best universal client storage pattern.

The only universal client storage API is localStorage in browser and AsyncStorage in React Native.

The best native client storage free API is IndexedDB in browser and SQLite in React Native.

There are several abstractions over IndexedDB and SQLite, like PouchDB or WatermelonDB, but all those abstractions have to preload the whole database to allow rich queries and save everything on any change.

Now something else.

The problem with browser localStorage is that it's blocking and does not work in Web Workers.

The problem with IndexedDB is that sooner or later we have to fetch all items to do things.

There is no trick to emulate a real generic logical AND in indexedDB other that applying a JS filter to the iteration. This is the best hit you can make.

https://www.codeproject.com/Articles/744986/How-to-do-some-magic-with-indexedDB

So now we know we have to fetch the whole client DB to be able to make rich queries. IndexedDB will not help us.

The suggested solution is:

• For the browser, use LocalStorage emulated via idb-keyval in Web Worker.
• For React Native, use AsyncStorage out of the main thread.

The last issue is costly JSON.stringify across the whole client state.

1) It belongs to web worker / React native thread. 2) JSON.stringify can be optimized via structural cache.

That's all.

[steida]

To respoke:

• We will use IndexedDB to emulate better LocalStorage for browsers.
• AsyncStorage for RN is good enough.
• We preload everything, that's fine even for several megabytes of data.
• Because everything is loaded, we can have any query we want.
• As for saving, it will happen in Web Worker. JSON.stringify can be optimized via structural sharing cache.

As a result, the app code will be much simpler.

[steida]

The last issue. Once app will have hundreds of thousands of tasks, there are two potential issues: 1) Maybe slow queries without indexes. 100 ms to filter 100 000 tasks by three string indexOf props. 2) JSON.stringify the whole DB on any change. Not sure whether structural cache will help.

TODO: Measure it.

[steida]

After vacation, I rethink it again and now I suppose windowed IndexedDB is the best solution 😂.

Two issues with key-value storage emulated in indexedDB

• Having to pass therefore serialize the whole DB.

Most browsers implement the structured cloning algorithm, which allows you to pass more complex types in/out of Workers such as File, Blob, ArrayBuffer, and JSON objects. However, when passing these types of data using postMessage(), a copy is still made. Therefore, if you're passing a large 50MB file (for example), there's a noticeable overhead in getting that file between the worker and the main thread.

https://www.html5rocks.com/en/tutorials/workers/basics/

• Potentially slow queries because in-memory has no indexes. IndexedDB is, well, indexed.

• Having loading everything can be bad because of the aforementioned structured cloning algorithm.

Windowed IndexedDB means we do not load and save everything.