search

google / leveldb

LevelDB is a fast key-value storage library written at Google that provides an ordered mapping from string keys to string values.
BSD 3-Clause "New" or "Revised" License
36.56k stars 7.83k forks source link

Can leveldb be configured to auto-compact regularly? #735

Open ghost opened 5 years ago

ghost commented 5 years ago

Hello. During testing it has been noticed that leveldb will auto-compact when a user regularly stores new data. It will not auto-compact after many deletions with infrequent writes. Is there a configuration that must be toggled to enable this, or is the user expected to manually compact on frequent deletes from a large db? Is there a way to increase the rate of compaction. Alternatively, is there a recommended method for deleting data such that compaction is not needed? Certain operations, like SeekToFirst() degrade in performance very significantly without regular compaction (when deleting the first element, of course).

originalsouth commented 5 years ago

Maybe #414 is somewhat relevant?

ghost commented 5 years ago

@originalsouth That is helpful, in the sense that it is one more source which indicates that compaction is triggered on a write/expansion event. I am referring to this: https://github.com/google/leveldb/issues/414#issuecomment-270507020

I would prize a definitive response, as I am dealing with a defect where performance degrades when you start with a full database and your goal is to consume/gradually remove all the elements. I am debating whether manual compaction should be performed at intervals.