search

mosuka / cete

Cete is a distributed key value store server written in Go built on top of BadgerDB.
Apache License 2.0
234 stars 28 forks source link

Add garbage collector for badger #48

Closed christian-roggia closed 4 years ago

christian-roggia commented 4 years ago

There is a very important feature missing from BagderDB: the garbage collector for database vlog files. We have experienced database outage due to "no space left on device" errors, an internal investigation led to the conclusion that the database vlogs were increasing at each restart and never collected, ultimately leading to the disk being completely filled with vlogs.

This PR introduces a garbage collector routine as described in https://dgraph.io/docs/badger/get-started/#garbage-collection

If merged this PR it can be further extended by add 2 new flags --gc-interval=5m --gc-discard-ratio=0.5 as well as a context that can be used to cancel the garbage collector.

mosuka commented 4 years ago

Thanks for the PR. I appreciate your contribution!