benbjohnson / litestream

Streaming replication for SQLite.
https://litestream.io
Apache License 2.0
10.36k stars 232 forks source link

Allow a restored database to pull changes from a central "write database". #450

Closed thestalnakergroup closed 1 year ago

thestalnakergroup commented 1 year ago

Apologies if this already exists. But, I would like the ability to auto-restore from the replication database. My use-case is:

I have one "write server" that runs a cron job to search through documents and store keywords related to a document in the sqlite database.

I then have multiple read servers that read from the restored sqlite database.

I have encountered an issue where I have to manually restore the database in order for the changes that have been stored in the write database to each of the read servers.

Is there a way we would allow the restore command to periodically reach out to S3 (or other storage) and pull those changes into the read database?

hifi commented 1 year ago

There was an incomplete live read replica mode for Litestream that was scrapped in favor of LiteFS and was considered out of scope after that.

benbjohnson commented 1 year ago

@hifi is correct. The read replication in Litestream was difficult to make work well since Litestream runs as a separate process.