Support for Cloud DB - Githubissues

YourAverageMoron commented 1 month ago

Rationale - I'd love to be able to easily share the nom db across my home and work laptop allowing me to sync my favourites and read items, there might be a more sensible way to do this but I was looking into Turso's embedded replicas and thought it might be an interesting approach

POC - I've started a rough implementation of this - see here - which allows you to add config for a turso db, you can see in the video we have 2 config folders (trying to mimic 2 laptops) and a turso db, start the test1 nom instance it populate the tables within Turso and a local replica in the test1 folder, then when we run test2 the data form Turso is replicated into the Turso db tables (sorry i didnt ls the test2 dir but it also gets a replica of the db - you'll have to trust me on that one). This may give an approach to sharing config easily between machines

turso_demo

Config changes:

db:
  turso:
    primaryUrl: <TURSO URL>
    authToken: <TURSO AUTH TOKEN>

This is nowhere near a finished product, it needs properly testing, and I want to improve the offline experience - currently it just hangs but with embedded replicas I think we should be at least able to view based off the local replica and maybe even update and sync once the client is back online

Apologies for the rambly Github issue, I'm going to continue working on this when I have chance but I would really like to hear your thoughts!

guyfedwards commented 1 month ago

Hey thanks for the issue. I'm happy for something like this to go in, having it behind a config option means that it doesn't break the current way of doing things and gives an alternative syncing solution.

Few things to think about in the full implementation:

Maintaining current default functionality
Migration of current to turso and back (if a user adds config for turso and then removes, what happens)
At a quick glance turso seem to have a non-cgo version of their client lib which would be great to use regardless as simplifies the build steps, https://github.com/tursodatabase/libsql-client-go
If you are able to figure out a nice way to write tests for this, that would be awesome

Look forward to seeing how you get on! Let me know if you have any questions

YourAverageMoron commented 1 month ago

Yep I completely agree with these points, in that POC i the turso config is omitted the it should just use the default local db (but its using turso's libsql)

Migration of current to turso and back - the turso to local/default migration works fine and just uses the local replica as the db and you keep all of your data. However i might need to look more into the local/default to turso migration, I have a feeling it wont sync the data from the local db to turso but instead will try and pull data from turso to the local db which will delete all of your existing data

And yeah i will take a look at the non-cgo lib and adding some testing

guyfedwards / nom

Support for Cloud DB #97