search

simolus3 / drift

Drift is an easy to use, reactive, typesafe persistence library for Dart & Flutter.
https://drift.simonbinder.eu/
MIT License
2.65k stars 372 forks source link

Support remote SQLite databases #3069

Closed exaby73 closed 4 months ago

exaby73 commented 4 months ago

Is your feature request related to a problem? Please describe. Turso is a "serverless"/"edge" database that uses SQLite (libsql). My request is to provide a way to connect to remote SQLite / libsql databases like Turso

Describe the solution you'd like Since the package already has SQLIte parsing, it could be used over a remote connection as well. This would be good as we could write Dart backends on Globe that would take advantage of Turso's distributed nature

simolus3 commented 4 months ago

I'm not that familiar with turso, but can libsql (as a C library) be used to connect to remote turso databases? That would make it easy to use with drift, since libsql is apparently compatible with libsqlite3 and package:sqlite3 (which drift uses internally) provides bindings to that.

dickermoshe commented 4 months ago

I'll take a quick look at the Python Client SDK...

dickermoshe commented 4 months ago

What turso did was add networking to sqlite. We wouldn't be using package:sqlite3 at all.

We would just need a websocket or http client. They created a protocol that could use either https://github.com/tursodatabase/libsql/blob/main/docs/HRANA_3_SPEC.md

We could use this to create a websocket client. The tricky part is parsing the returns correctly.

There are multiple clients that are available to port from.

dickermoshe commented 4 months ago

Straight from the horses mouth

https://x.com/glcst/status/1806414220902215981 https://x.com/mickrich384/status/1806534564124201389 https://x.com/glcst/status/1806535191818641460

If they pull through on a Dart client/server, we could use it to replace package:sqlite3 too !

simolus3 commented 4 months ago

It looks like a pretty simple RPC protocol so it shouldn't be super hard to implement. I'm not interested in anything Turso-specific, but I'll take a look at implementing a hrana client with drift support.

dickermoshe commented 4 months ago

I'm not interested in anything Turso-specific

There is one major benefit to Turso. It's not a synchronous library. It wouldn't need an Isolate. For Dart Native, using an Isolate is good practice anyway. But for Dart Web it would be a major help. No dealing with Javascript workers.

I remember reading this somewhere, trying to find it now.

simolus3 commented 4 months ago

There is one major benefit to Turso. It's not a synchronous library. It wouldn't need an Isolate.

I meant that I'm happy to write bindings to servers following the open-source Hrana protocol (which is asynchronous in nature, we're using websockets to send commands), but not for the Turso SDK responsible for giving us a connection URL and JWT token to connect to.

Anyway, here's drift_hrana, fresh from the press. I'll add a page to the documentation as well, but there's an example in the package too. You only have to swap out the NativeDatabase with a HranaDatabase to use a server, the rest of drift behaves exactly the same.

dickermoshe commented 4 months ago

You're an absolute beast!