search

benbjohnson / litestream

Streaming replication for SQLite.
https://litestream.io
Apache License 2.0
11.1k stars 256 forks source link

Allow setting replica interval offsets #523

Open hifi opened 1 year ago

hifi commented 1 year ago

If using S3 replicas the extra LIST call on startup may be expensive in scale if a lot of Litestreams are started in quick succession.

Allowing configuring the snapshot offset externally allows the caller to spread around the snapshots without relying on access to the remote replica making restarts no-op.

Same thing applies to retention checks that they can easily stack too close.

hifi commented 1 year ago

I'll modify this a little as we need an offset for retention check as well.

hifi commented 1 year ago

Expanded to cover all intervals. We use both snapshot and retention check offsets to prevent storming a bucket with requests around the same time.