tecbot / gorocksdb

gorocksdb is a Go wrapper for RocksDB
http://rocksdb.org
MIT License
937 stars 269 forks source link

Can't create a valid WAL iterator #192

Closed hkparker closed 4 years ago

hkparker commented 4 years ago

In the following trivial example, I attempt to get a WAL iterator on a database using the latest sequence number.

package main

import (
    log "github.com/sirupsen/logrus"
    "github.com/tecbot/gorocksdb"
)

func main() {
    opts := gorocksdb.NewDefaultOptions()
    opts.SetCreateIfMissing(true)
    testDB, err := gorocksdb.OpenDb(opts, "test")
    if err != nil {
        log.WithFields(log.Fields{
            "error": err.Error(),
        }).Error("error opening database")
        return
    }

    lastSequence := testDB.GetLatestSequenceNumber()
    walIter, err := testDB.GetUpdatesSince(lastSequence)

    if err != nil {
        log.WithFields(log.Fields{
            "error":    err.Error(),
            "sequence": lastSequence,
        }).Error("error getting WAL iterator")
        return
    }
    if walIter == nil {
        log.WithFields(log.Fields{
            "sequence": lastSequence,
        }).Error("WAL iterator is nil")
        return
    }
    if walIter.Err() != nil {
        log.WithFields(log.Fields{
            "error":    walIter.Err(),
            "sequence": lastSequence,
        }).Error("WAL iterator has an error")
        return
    }
    if !walIter.Valid() {
        log.WithFields(log.Fields{
            "sequence": lastSequence,
        }).Error("WAL iterator is not valid")
        return
    }
}

Running this code produces the following:

ERRO[0000] WAL iterator is not valid                     sequence=0

I've also done more complex examples, where I've loaded some data and the sequence is not 0, but I still get an invalid iterator.

Unless I'm missing something, this seems like a bug in gorocksdb, or rocksdb itself? I'm using arch to install rocksdb, which brought down version 6.5.3-1. Gorocksdb built just fine, but based on this I'm thinking maybe it's a version issue. I'll try to build on an 5.X version and see what happens (though it would be great to be on >= 6.2.2 so I don't have to backport the fix for the memory leak).

hkparker commented 4 years ago

Reproduced this in an ubuntu docker container using librocksdb-dev, which is version 5.17.2-3

hkparker commented 4 years ago

Seems like it's invalid if there's no data in it. A write just before getting the WAL iterator results in a valid iterator.