search

tekartik / sembast.dart

Simple io database
BSD 2-Clause "Simplified" License
780 stars 64 forks source link

Need a way to reload the database / detect that it was changed by an external process. #374

Open danteCarvalho opened 7 months ago

danteCarvalho commented 7 months ago

I have a flutter app and a windox/linux dart service that uses the same sembast database. The service starts when the OS boots and run every 30 minutes.

If they are both running, any change to the database may not be reflected on the other, it seems like a new value wont show to the other as it is already loaded in memory.

1 One way to do this is to reload the database before every read/write operation. Maybe with a boolean param 'forceReload' on the store.record and store.find methods or on the openDatabase method. (maybe put a warning to not use with a big database)

2 Or i can just close and open the database every read/write operation, but that seems ugly.

3 The database could reload by itself when it detects that it was changed by an external process

alextekartik commented 7 months ago

sembast regular io format is not cross-process safe. A safer solution would be to use sembast_sqlite and sqflite_common_ffi which provides the same behavior and interface but is cross process safe (i.e. a record saved in an application is visible right away in the other application) although listeners (onSnapshot) won't get notified. This requires the sqlite library to be present (on linux and windows) and the storage format will change.

danteCarvalho commented 7 months ago

i tried now with sembast_sqlite and sqflite_common_ffi but await store.find(db!, finder: finder); still returns old values, 1 suppose there is nothing on the db 2 i add one object on the app, 3 the service that is on loop try to get all the values, but it is still empty 4 if i restart the service the new value appears

alextekartik commented 6 months ago

Thanks for the report @danteCarvalho and sorry for the delay. Indeed the database was currently only checked upon writing so any read would return the previously known data until a write is made.

I added in sembast 3.7.1 the helper database.checkForChanges() that should properly re-read the storage. Doing this for every read has a significant performance impact as it will read a database information (just an int) to check for potential changes so for now I'm reluctant to do it for every read but we could imagine setting a cross-process mode that will call this everytime.

Does this solve (partially) your issue?

danteCarvalho commented 5 months ago

Oh, thanks for the update, i will try it and give some feedback.