tursodatabase / libsql

libSQL is a fork of SQLite that is both Open Source, and Open Contributions.
https://turso.tech/libsql
MIT License
8.62k stars 229 forks source link

The story around backups to S3 is unclear #1277

Open jazoom opened 3 months ago

jazoom commented 3 months ago

The libsql SDK docs don't mention backups to S3, but it is mentioned in this blog post:

https://turso.tech/blog/fully-open-source-encryption-for-sqlite-b3858225

So after features like native replication, automatic backups to S3 and a serverless mode, we are adding to libSQL yet another feature that is critical to production workloads: encryption at rest.

The article goes on talking about the libsql client.

const db = createClient({
  url: 'file:sqlite-enc.db',
  encryptionKey: process.env.ENCRYPTION_KEY,
});

As far as I can tell, sending encrypted backups to S3 like this is not supported.

I can see there is something called "bottomless" for libsql-server.

https://github.com/tursodatabase/libsql/tree/main/libsql-server

But that doesn't apply if one is using a local file, such as in the blog post, correct?

I'd like something like Litestream, for an encrypted local file libsql database.

haaawk commented 3 months ago

This is encryption at rest of embedded replica stored on the client. Backups to S3 are server feature. They are similar to litestream.

jazoom commented 3 months ago

It might be worth making that more clear in your docs and blog post?

Litestream works on a local file SQLite database.

haaawk commented 3 months ago

FYI @notrab