DallasHoff / sqlocal

SQLocal makes it easy to run SQLite3 in the browser, backed by the origin private file system.
https://sqlocal.dallashoffman.com
MIT License
212 stars 8 forks source link

Node support #3

Closed fabiospampinato closed 11 months ago

fabiospampinato commented 11 months ago

There seems to be a origin-private file-system ponyfill, it'd be cool if it could be used to make this module isomorphic and work everywhere with good performance, with the same APIs, and without ever touching native node modules.

DallasHoff commented 11 months ago

My goal with this library is to make it easier to use the WASM build of SQLite in the browser, which is a relatively new thing. On the other hand, running SQLite on the server side is pretty much a solved problem and a very different use-case. There's already lots of ways to use the standard/server build of SQLite, so I don't think supporting Node with SQLocal would really contribute anything. Using WASM with polyfilled file system access seems like a lot of unnecessary overhead when Node can already just run the native version of SQLite and get better performance since it doesn't have the same restrictions that a browser environment has.

fabiospampinato commented 11 months ago

There's already lots of ways to use the standard/server build of SQLite, so I don't think supporting Node with SQLocal would really contribute anything.

A WASM build would be way easier to handle, like getting native modules to work in Electron across platforms is a total nightmare. You can just spawn the sqlite3 binary, but then Apple's notarization won't like you very much, and you'll lose a significant amount of performance due to serializing things back and forth.

DallasHoff commented 11 months ago

If it's Electron we're talking about, there's no reason, as far as I know, that you wouldn't be able to use SQLocal as is. You'd just run it right in the renderer process. No Node required.

fabiospampinato commented 11 months ago

Maybe you want to manage a sqlite database from the main process, maybe you want to open the database once there and provide its data to multiple open windows potentially.

DallasHoff commented 11 months ago

You should be able to connect to the same OPFS database from multiple windows. I feel like that would be easier than passing data back and forth between the main process and the renderer processes.

fabiospampinato commented 11 months ago

It's just an example scenario that goes against this:

I don't think supporting Node with SQLocal would really contribute anything

Here's another scenario: you need sqlite in Node, and you run some tests. Then you spawn a VM and run the same tests there, or you run the same tests under a different version of Node. You could use better-sqlite3 for this, but you'd have to wait for the native modules to be rebuilt each time you switch between environments. You can't use an FFI version of sqlite because Node doesn't support it. You could spawn native sqlite3 binaries but then serialization costs might be prohibitive for what you are doing.

Here's another one: you want to build a package that runs in both Node and Deno. Now you can't use better-sqlite3 anymore, at least for Deno, I think.

A better solution here, or at least the one I would use, is just to use a WASM build of sqlite, which doesn't require recompilations, doesn't require dealing with notarization for custom builds, and doesn't have a big serialization overhead either. It's kinda like the best of all worlds, if one doesn't need WAL mode.

So there is greater than no utility to having something like that.

But it's completely fine if the scope of this package is narrower, of course 👍