LegendApp / legend-state

Legend-State is a super fast and powerful state library that enables fine-grained reactivity and easy automatic persistence
https://legendapp.com/open-source/state/
MIT License
3k stars 87 forks source link

[Question] SQLite Persist Plugin? #339

Open iykazrji opened 3 months ago

iykazrji commented 3 months ago

Just came across this package and I'm very impressed! Great work on this!

Wanted to know if there are plans to create a persist plugin for Sqlite similar to what we have for MMKV? This would offer a lot of benefits especially for Local first mobile apps.

Thanks!

realPrimoh commented 1 month ago

From the developer on Reddit:

Legend-State developer here. The other important thing to think about is how you use storage, whether you're using a few keys or a lot of keys. It's a bit more complicated than one being strictly better, there's some tradeoffs.

My understanding of why AsyncStorage can be slow is that In iOS AsyncStorage opens and reads a file for each call to getItem, and for each key passed to multiGet (1). And in Android it uses SQLite but it runs a separate query for each key (2). It does not hold anything in memory, so subsequent calls to getItem load from the file again.

So if you only have a few keys it shouldn't be too slow, but it adds up if you use a lot of keys. So if you're loading and saving a small number of keys then AsyncStorage should be fine. If you load from getItem or multiGet once upfront that should be ok, but if you use it a lot be aware that it's not cached in memory and reads a file every time.

If you're using it through Legend-State's persistence system then it'll save each observable as a key in AsyncStorage (with the name parameter). And it only reads once upfront, then it's saved in memory in the observable. So if you don't have a huge number of observables, then that should be fine. As others said, for a comparatively small amount of data then MMKV will likely be faster, but once data gets really big it's possible it might end up slower since it's synchronous. But it depends largely on usage. I've personally been happy using MMKV in my apps.

For what it's worth, I did a bunch of benchmarking and testing a while ago and found that for my app's usage of loading tens of thousands of rows of items, SQLite was even slower than AsyncStorage. I ended up just reading/writing to JSON files and that was orders of magnitude faster than selecting a huge number of rows from SQLite.

But the other question is whether you want to load everything upfront or only a subset. If you want to load only a subset then being able to query SQLite makes it a great option. But if you're loading everything up front then you're not really getting the benefit of a database and I would guess it's likely slower than other options.

By the way, one goal of Legend-State is to make it easy to change persistence layers. So when you want to migrate you could syncObservable with the new persistence, and if it's empty then syncObservable with the old persistence. If there's something loaded from the old observable then just set it onto the new observable to save it to the new persistence, and it will load from that one in the future.

This got longer than I expected 😅. But hope it helps!

(1) https://github.com/react-native-async-storage/async-storage/blob/f790cb3e6e0db0338fcfde13d94a768c0e7c5eef/packages/default-storage/ios/RNCAsyncStorage.mm#L590 (2) https://github.com/react-native-async-storage/async-storage/blob/f790cb3e6e0db0338fcfde13d94a768c0e7c5eef/packages/default-storage/android/src/main/java/com/reactnativecommunity/asyncstorage/AsyncStorageModule.java#L114

I do think it would be a good idea, but I also think it'll probably come in the form of a community contribution (which you should definitely work on :))