v2: Constant CPU usage

[axllent]

Hi @ostafen. As you would probably have seen (in Mailpit), a user reported a constant CPU usage, which I believe I have traced back to CloverDB v2 (both 1 & 2 alpha).

A simple test:

package main

import (
    "time"
    "github.com/ostafen/clover/v2"
)

func main() {
    db, _ := clover.Open("", clover.InMemoryMode(true))
    defer db.Close()
    time.Sleep(600 * time.Second)
}

Run the program and check CPU usage with top/htop - the running binary will be using a constant 2-3% CPU. This does not happen with CloverDB v1.

Any ideas?

[ostafen]

Hi, @axllent. I don't see any clear reason for this to happen inside clover's code, since Open() simply creates a badger db instance and starts a background goroutine for clean-up (which runs every 5 minutes).

func (s *storageImpl) startGC() {
    s.chWg.Add(1)

    go func() {
        defer s.chWg.Done()

        ticker := time.NewTicker(s.conf.GCReclaimInterval)
        defer ticker.Stop()

        for {
            select {
            case <-s.chQuit:
                return

            case <-ticker.C:
                err := s.db.RunValueLogGC(s.conf.GCDiscardRatio)
                if err != nil {
                    log.Printf("RunValueLogGC(): %s\n", err.Error())
                }
            }
        }
    }()
}

func (s *storageImpl) Open(path string, c *Config) error {
    if c.InMemory {
        path = ""
    }

    db, err := badger.Open(badger.DefaultOptions(path).WithLoggingLevel(badger.ERROR).WithInMemory(c.InMemory))

    s.db = db
    s.conf = c

    s.startGC()

    return err
}

[axllent]

But you can confirm the constant CPU usage on your end though, even using that simple test? Maybe it's the badger version then? 2-3% CPU isn't the end of the world, however it should be 0 for most of the time (when obviously not reading/writing/purging).

[ostafen]

Yes, I confirm that. However, the badger version is the same as v1.2.0

[axllent]

@ostafen I can confirm that I get the same constant 2-3% CPU usage when running badger directly (default options + in memory). I had done a complete comparison, only to realize later that v1 of CloverDB doesn't use badger for the in-memory database at all whereas v2 does. I guess that explains why v1 (in memory) doesn't have that issue then :)

I'm suspecting there isn't much, if anything, you can do here as it appears to be directly related to badger?

[ostafen]

Unless this is due to something that can be configured when opening the badger database, there's nothing I can do