search

penberg / limbo

Limbo is a work-in-progress, in-process OLTP database management system, compatible with SQLite.
MIT License
896 stars 49 forks source link

Disallow multiple processes opening same database file #94

Closed penberg closed 3 weeks ago

penberg commented 1 month ago

SQLite uses POSIX advisory locks for every transaction to allow multiple processes to write to the same database file. This leads to every read taking micro-seconds, which is not what we want. However, we do need to disallow multiple processes from opening the same database file to avoid corrupting the file.

gvos94 commented 1 month ago

hey @penberg. Will take a crack at this!

penberg commented 1 month ago

Nice @gvos94! 👏👏👏

edude03 commented 1 month ago

How are you thinking about solving @gvos94? My naive thought would be to write the process ID and lock time to some part of the file.

penberg commented 1 month ago

POSIX advisory locks are the way to do it. The OS does the locking for you

gvos94 commented 1 month ago

@penberg Should we also prevent multiple threads from opening the same database file to avoid potential corruption? I'm unsure how concurrent writers would operate in that scenario (https://github.com/penberg/limbo/issues/86). Do you expect concurrent writers to share the same DB handle?

Additionally, since we're disallowing multiple processes from opening the same database file, is the idea to only support concurrent writers within the same process?

penberg commented 3 weeks ago

Fixed by https://github.com/penberg/limbo/pull/151