Syncing between two StorageEngines

[pnewell]

Are there any examples patterns for building an app that actually does sync data remotely when online? IE store data in a local sqlite storage engine that is also automatically synced remotely? If I were to build that myself, is there a recommended pattern to use? Is it two separate StorageEngines? How do they interact together?

[mergesort]

Hey @pnewell, thanks for the good question. If you don't mind I'm going to answer #46 and #47 together here since they're actually similar and related questions.

The short answer is that I don't have a great answer for this. In my app Plinky I have two targets, the main app and a share extension. A user can save a link from the share extension, and naturally they expect to see that link appear in the app.

To accommodate this I have this function run, listening to the app's foreground notification to know to fire.

func synchronizeExternallySavedLinks() async {
    do {
        // We create a temporary ObjectStorage to retrieve any links that were saved using the share extension.
        // Since the app's $links memory store does not get updated directly, when the app is launched post-save
        // it will not display the link that was saved in another target.
        //
        // To work around this we create an ObjectStorage based on the same database, pull the links out of there
        // and then re-save them. This is a somewhat clever shortcut to synchronizing the in-memory store since
        // the data is already saved on disk.
        //
        // Many approaches using multiple stores were tried, but left us with states that were highly unsynchronized.
        // I tried to manually synchronize them, but would run into conflicts when going between the main app
        // and another target (such as having saved a link in Safari).
        let objectStorage = ObjectStorage<RichLink>(storage: SQLiteStorageEngine.linksStorageEngine)
        try await self.$links.insert(objectStorage.allObjects())
    } catch {
        print("Failed to synchronize links", error)
    }
}

Now you may be saying, Joe, this doesn't look very good. And friend, I would be very inclined to agree with you. As you suggested in #47, I'm not opposed to having a re-synchronization process. One of the core principles of Boutique is data consistency, so I don't want changes to occur automatically in a way where the user isn't aware changes occurring. This mostly leaves me with pursuing a manual approach, one that's triggered infrequently, somewhat like what I came up with above.

I may not have a good answer, but what I am is very open to suggestions, ideas for patterns, and other thoughts you may have on the subject. Hope that helps!