search

sqlpage / SQLPage

Fast SQL-only data application builder. Automatically build a UI on top of SQL queries.
https://sql.datapage.app
MIT License
1.57k stars 89 forks source link

Private s3 storage for SQLite db #578

Closed Vladimir-Va closed 1 month ago

Vladimir-Va commented 1 month ago

Hello SQL pages team! First of all, thank you for your work, it is at least very interesting and unusual. I'm using a tool Grist for the development of sqlite databases. In my opinion, SQl Pages will go well with this tool. Grist same as Litestream can push latest version of database to private S3 bucket. Please consider connecting to s3 private repositories for streaming sqlite databases as datasources for SQL Pages.

lovasoa commented 1 month ago

Hi Vladimir, and welcome to SQLPage!

Is there something specific preventing you from using Litestream directly? It’s indeed a great tool and should work well with SQLPage. If you're having trouble integrating the two, I’d be happy to help!

Vladimir-Va commented 1 month ago

Thanks for the answer, I really don't see any problems for the URL from pulic bucket, but unfortunately I don't know how to get data from a private bucket. We need to use credentials. I wanted to suggest considering using the built-in s3 client in sql pages. I'm sorry if the question is stupid, I just started to figure it out.

lovasoa commented 1 month ago

Litestream does already work with private s3 buckets. Can you follow the tutorial at https://litestream.io/getting-started/ and tell me where it doesn't work for you, if you encounter an issue ?

Vladimir-Va commented 1 month ago

Ow, i see now you mean Restoring your database But i mean direct connecting to url s3://mybkt.localhost:9000/fruits.db directly from SQLPages app, if is it possible ofcourse.

lovasoa commented 1 month ago

Your database will be downloaded from s3 on start, then continuously streamed back to it, which is fast and failure-proof.