Out-of-range panic in updateOffset

fjl commented 7 years ago

I get this crash when running my program, which embeds badger @ bc04380.

panic: runtime error: index out of range

goroutine 55 [running]:
github.com/ethereum/go-ethereum/vendor/github.com/dgraph-io/badger/badger.(*KV).updateOffset(0xc420107680, 0x0, 0x0, 0x0)
    /Users/fjl/develop/eth/src/github.com/ethereum/go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/dgraph-io/badger/badger/kv.go:316 +0x10c
github.com/ethereum/go-ethereum/vendor/github.com/dgraph-io/badger/badger.(*KV).writeRequests(0xc420107680, 0xc42560beb8, 0x1, 0xa)
    /Users/fjl/develop/eth/src/github.com/ethereum/go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/dgraph-io/badger/badger/kv.go:392 +0x407
github.com/ethereum/go-ethereum/vendor/github.com/dgraph-io/badger/badger.(*KV).doWrites(0xc420107680, 0xc42013a450)
    /Users/fjl/develop/eth/src/github.com/ethereum/go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/dgraph-io/badger/badger/kv.go:420 +0x2b7
created by github.com/ethereum/go-ethereum/vendor/github.com/dgraph-io/badger/badger.NewKV
    /Users/fjl/develop/eth/src/github.com/ethereum/go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/dgraph-io/badger/badger/kv.go:152 +0x593

To reproduce:

git clone --branch badger-exp https://github.com/fjl/go-ethereum 
cd go-ethereum
make
./build/bin/geth --badgerdb --datadir=DATA

geth is a client for the Ethereum cryptocurrency network and is probably a really good smoke test for your KV store because it's RW heavy. When you run it, blockchain data will be downloaded from the p2p network and stored in badger (at least that's what should happen, it doesn't really though because of the crash). The commit that adds experimental support for badger is https://github.com/fjl/go-ethereum/commit/a7f7238c3f91f43e7f357aead60617091db74f76.

igm commented 7 years ago

Probably related as this produces the same runtime error:

package main

import (
    "github.com/dgraph-io/badger/badger"
)

func main() {
    kv := badger.NewKV(&badger.DefaultOptions)
    kv.BatchSet([]*badger.Entry{})
    kv.Close()
}

manishrjain commented 7 years ago

Thanks for reporting. Will look at it first thing tomorrow.

fjl commented 7 years ago

I'll try avoiding the call to BatchSet if length is zero.

manishrjain commented 7 years ago

Thanks for the code, @igm. I just pushed a fix to master. You can try it out now @fjl.

fjl commented 7 years ago

It doesn't crash anymore.

manishrjain commented 7 years ago

Eager to hear your experience with Badger + Ethereum. I'll try and run it as well.

manishrjain commented 7 years ago

Running it. It seems a bit slow.. I think there might not be enough batching going on. I'd create batches of 1000 entries each to speed it up.

dgraph-io / badger

Out-of-range panic in updateOffset #24