msiemens / tinydb

TinyDB is a lightweight document oriented database optimized for your happiness :)
https://tinydb.readthedocs.org
MIT License
6.75k stars 536 forks source link

[IDEA] lock the file when database is opened. #219

Closed Cologler closed 6 years ago

Cologler commented 6 years ago

Since tinydb cannot access from multiple processes or threads, I think this is one way to keep the data safe.

reference:

https://github.com/WoLpH/portalocker

msiemens commented 6 years ago

You can implement this in your own Storage implementation πŸ™‚ With the current architecture it's not possible to write a mechanism which does file locking for all possible Storage implementations as Storages might use an in-memory storage or even storing data on the network, where file locking won't make any sense

Cologler commented 6 years ago

JSONStorage is file system storage, the lock can implement inside it.

Markus Siemens notifications@github.com 于 2018εΉ΄8月3ζ—₯周五 21:36ε†™ι“οΌš

You can implement this in your own Storage implementation πŸ™‚ With the current architecture it's not possible to write a mechanism which does file locking for all possible Storage implementations as Storages might use an in-memory storage or even storing data on the network, where file locking won't make any sense

β€” You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/msiemens/tinydb/issues/219#issuecomment-410249126, or mute the thread https://github.com/notifications/unsubscribe-auth/AKZtUmuAXIqZ7IceQ9cw4EOgIa6TevmBks5uNFHVgaJpZM4VkTl- .

msiemens commented 6 years ago

That's correct. If you want, you can create such a storage as a separate repository and I can link it from the extensions list πŸ™‚ Although I don't want to integrate this into the TinyDB core as it's another performance penalty for users who use single threaded database access.

Cologler commented 6 years ago

I don't think this has performance penalty since only lock file once in JSONStorage.__init__().

The reason I have not create a PR for this is the lock require a external package portalocker, but the tinydb readme says: has no external dependencies.

msiemens commented 6 years ago

I don't think this has performance penalty since only lock file once in JSONStorage.init().

In this case only one TinyDB instance could be open for the file at a time. In other words, as long as the file is opened at all, no other TinyDB instance can access it. If you want to allow interleaved access, you'll have to use the lock on every read and write operation with the overhead I already described.