search

jlongster / absurd-sql

sqlite3 in ur indexeddb (hopefully a better backend soon)
MIT License
4.15k stars 101 forks source link

WIP webkitFileSystem backend #13

Open jlongster opened 3 years ago

jlongster commented 3 years ago

This adds a backend which uses webkitFileSystem. Because it provides a sync API, it is much simpler. It's also way simpler because we have seemingly random access to the file, so we don't have to worry as much about optimizations and cursors and such.

However, there is a crucial problem: it provides no locking mechanism. If we don't have that, or any way to atomically check and swap a byte, then this will never be viable. Without that there's no way to tell if it's safe to write or not when there are multiple connections.

I didn't get that far though: this still isn't working fully. If you do yarn serve --config-name bench and open http://localhost:8080/ in Chrome, hit "write a little data" and then "sum all". it will work! but if you try "write a lot of data" and then "sum all" you get a "db is malformed" error. I don't know why it fails with larger data, but that must mean there's some kind of race condition even though we are using the sync API. Anyone want to try to fix it?

Note that you can delete the file by clicking "delete file" in the demo page, which helps restart it from scratch after you get that error. "Delete file" does nothing if you haven't opened the file yet; on first load of the page make sure to click "sum all" or something to open it, then "delete file" to delete it. Need to fix that.

chuliomartinez commented 2 years ago

Why are locks needed? Maybe i remember it wrong, but sqlite has a compile option to support multiple threads. Isn’t javascript single threaded anyways? Or is just to support multiple open tabs scenarios?

quolpr commented 2 years ago

Multiple open tabs scenario I guess

chuliomartinez commented 2 years ago

Can’t we use the fa layer for locks? Like a “lock” file? I think a file can only be exclusively opened right? That would work for write/exclusive locks. Read lock could be a open file, read number, increment/decrement, close file. With busy wait. What do you think?