search

joaoh82 / rust_sqlite

SQLRite - Simple embedded database modeled off SQLite in Rust
MIT License
1.07k stars 59 forks source link

Using WASM instead of custom virtual machine? #6

Open vlovich opened 3 years ago

vlovich commented 3 years ago

I was actually toying around with building a Rust database not too long ago & the interesting thought I had was about using WASM w/ a custom API instead of a VM to execute queries. The mental model I had was one of LLVM IR - that would make this suitable as a general database layer that's potentially accessible from any language. The JIT part of WASM is a nice "free" feature you get although I don't think it would meaningfully add performance but I think having a language-agnostic target API to build against could be interesting, letting any language target their queries against the runtime (i.e. write the database code in the language of your choice with domain-specific optimizations rather than using plain text SQL).

The open design question I didn't explore is what does the API look like & do you put the plan optimizer as part of the SQL layer or if you have the WASM API be the plan optimizer. I think the latter is probably the easiest to bring up while the former offers the largest opportunity for exploring greenfield ideas in this space.

I know there's exploration of integrating WASM for stored procedures, but you would get that implicitly for free if it was WASM at the lowest layers.

On the other hand, this could be a terrible idea. Wonder what your thought on this architecture is.

joaoh82 commented 3 years ago

I like this idea. Let's explore a bit more. About the open design question I would put the plan optimizer as part of the SQL layer instead in the WASM API. It seems to be right place for it. Also like you said, it would offer a largest opportunity to explore greenfield ideas, which I am always for it.

I always wanted to explore WASM a bit more. So if you are up for it, let's do it.

vlovich commented 3 years ago

I'm down to explore. I'm a decent engineer but have absolutely no background designing databases for real, so I think the best results might come from pairing with someone with some experience who can handle the design aspect & I can just be the code monkey to start with.

joshbenaron commented 3 years ago

Hey, it would be great if you could somehow compile sqlrite down to a wasm file. I work on a distributed compute blockchain which only runs wasm, and frankly C is just painful to work with. I'm happy to help out with this

joaoh82 commented 3 years ago

That is actually a really good idea.

joaoh82 commented 3 years ago

Btw, I created a discord server for the project so we can start have a more active conversation and build a community. Here is the link: https://discord.gg/zHXUG66X .