search

superfly / litefs

FUSE-based file system for replicating SQLite databases across a cluster of machines
Apache License 2.0
3.76k stars 90 forks source link

VFS #165

Open benbjohnson opened 1 year ago

benbjohnson commented 1 year ago

LiteFS should work as a SQLite VFS so that non-Linux users and users who don't like FUSE can also run it.

mrkurt commented 1 year ago

I would really love for this to work with Deno. Is a vfs pretty much going to have to be rust?

anacrolix commented 1 year ago

That's a good point, the performance overhead calling Go to C is high, and C to Go is incredibly high.

benbjohnson commented 1 year ago

I would really love for this to work with Deno. Is a vfs pretty much going to have to be rust?

@mrkurt I was thinking that the WASM version would have more limited functionality since a lot of those runtimes don't have a persistent disk so they'll probably just act as a thin client to a separate storage service. If that's the case, then we're stripping out a lot so I'm actually leaning towards making that client in TypeScript instead of Rust.

The VFS I'm thinking of for this issue would just be a rebundling of the Go code without FUSE and targeted at a single database connection.

nickchomey commented 1 year ago

I'm not sure if this will be helpful here, but rqlite implemented some vfs stuff in this PR, which was based on some VFS nuances explained in a different thread that is linked in that PR.

nickchomey commented 1 year ago

Perhaps this goes too far into the weeds, but this is an interesting paper that deeply explores FUSE performance as well as some optimizations. It sounds like it can have minimal or enormous performance lag depending on the application - sounds like the latter has been observed in litefs and that a vfs implementation would be worthwhile.

https://dl.acm.org/doi/fullHtml/10.1145/3310148

burggraf commented 6 months ago

What's the current status of VFS support?