search

google / leveldb

LevelDB is a fast key-value storage library written at Google that provides an ordered mapping from string keys to string values.
BSD 3-Clause "New" or "Revised" License
36.6k stars 7.83k forks source link

Make background thread stoppable #863

Open nshishov opened 3 years ago

nshishov commented 3 years ago

Hi,

Current version of LevelDB has detached background thread that can't be stopped and each environment calls std::abort() in destructor. This behavior prevents using of LevelDB in plugin dlls that can be started and stopped when main application still runs.

I think it is better to allow background thread to finish tasks in queue and then exit, when environment is being destroyed. That will make database loadable and unloadable.

pwnall commented 3 years ago

Thank you for the clear explanation!

We may be willing to consider a PR that fixes this problem, depending on the complexity it introduces in the codebase.

If we end up not accepting the fix, you can still use it in your own codebase. LevelDB is designed to work with custom Env implementations. The ones supplied in the project are intended to be good enough for most cases, and can be used as starting points when you need to write your own.

dlannan-fmad commented 3 years ago

Hi. Just wondering if this might make it work? We are using C api. Use leveldb_create_default_env(); to create a default env. Then remove the environment with leveldb_env_destroy(leveldb_env_t*) which would correctly destroy the Env object? And then this could be repeated? The problem I see that could occur, is the BackgroundMainThread may close during operations? Like compaction? Im still learning my way thought leveldb, so let me know if I have this all wrong.

ED I just tried this, and it seems to work ok. Will test a bit more.

ED Still seeing some problems with BachgroundMain thread. Doesnt look like a valid solve. Ok. I take that back. I forgot to delete options objects I had created. So, it does appear to be working. Will put up a little sample here.

To be clear we are doing:

  1. C API opening of leveldb objects (with options and various settings)
  2. Iterating, filling the db
  3. closing the db
  4. Opening the same db (sometimes in different thread)
  5. Iterating db
  6. closing the db

operations 4,5,6 may occur sequentially numerous times. And we were seeing issues with BackgroundMain thread seg faulting during compaction after a close.

For a dll, you should be able to do the above as well. Hope this helps.