absolute8511 / ZanRedisDB

Yet another distributed fault-tolerant key-value database Compatible with Redis written in Golang.
MIT License
74 stars 77 forks source link

负载均衡的时候kv进程core掉 #52

Open AlexStocks opened 6 years ago

AlexStocks commented 6 years ago

在测试系统负载均衡的时候,kv进程core掉,callstack如下:

0 runtime.raise () at /usr/local/go/src/runtime/sys_linux_amd64.s:113

1 0x000000000044e5d2 in runtime.raisebadsignal (sig=11, c=0x7f4e517f7ed8) at /usr/local/go/src/runtime/signal_unix.go:453

2 0x000000000044ea33 in runtime.badsignal (sig=11, c=0x7f4e517f7ed8) at /usr/local/go/src/runtime/signal_unix.go:562

3 0x000000000044e1eb in runtime.sigtrampgo (sig=11, info=0x7f4e517f80b0, ctx=0x7f4e517f7f80) at /usr/local/go/src/runtime/signal_unix.go:282

4 0x0000000000469b83 in runtime.sigtramp () at /usr/local/go/src/runtime/sys_linux_amd64.s:265

5

6 0x00007f4e5bad90d0 in std::string::assign(std::string const&) () from /lib64/libstdc++.so.6

7 0x00007f4e5bf95a64 in operator= (__str=..., this=0x7f4e517f8670) at /usr/include/c++/4.8.2/bits/basic_string.h:547

8 operator= (this=) at ./db/dbformat.h:170

9 rocksdb::CompactionPicker::GetRange (this=0x7f4da0a288c0, inputs=..., smallest=smallest@entry=0x7f4e517f8660, largest=largest@entry=0x7f4e517f8670) at db/compaction_picker.cc:152

10 0x00007f4e5bf9c1c8 in PickFileToCompact (this=, this=, this=, this=, this=) at db/compaction_picker.cc:1357

11 SetupInitialFiles (this=) at db/compaction_picker.cc:1104

12 PickCompaction (this=0x7f4e517f8720) at db/compaction_picker.cc:1216

13 rocksdb::LevelCompactionPicker::PickCompaction (this=, cf_name=..., mutable_cf_options=..., vstorage=, log_buffer=) at db/compaction_picker.cc:1400

14 0x00007f4e5bf7235c in rocksdb::ColumnFamilyData::PickCompaction (this=this@entry=0x7f4da0a27ee0, mutable_options=..., log_buffer=log_buffer@entry=0x7f4e517f91f0) at db/column_family.cc:812

15 0x00007f4e5bfcb6f4 in rocksdb::DBImpl::BackgroundCompaction (this=this@entry=0x7f4da0a3fd40, made_progress=made_progress@entry=0x7f4e517f8fce, job_context=job_context@entry=0x7f4e517f8ff0, log_buffer=log_buffer@entry=0x7f4e517f91f0,

prepicked_compaction=prepicked_compaction@entry=0x0) at db/db_impl_compaction_flush.cc:1526

16 0x00007f4e5bfce1f4 in rocksdb::DBImpl::BackgroundCallCompaction (this=this@entry=0x7f4da0a3fd40, prepicked_compaction=prepicked_compaction@entry=0x0, bg_thread_pri=bg_thread_pri@entry=rocksdb::Env::LOW) at db/db_impl_compaction_flush.cc:1343

17 0x00007f4e5bfce6bf in rocksdb::DBImpl::BGWorkCompaction (arg=) at db/db_impl_compaction_flush.cc:1162

18 0x00007f4e5c1170b8 in rocksdb::ThreadPoolImpl::Impl::BGThread (this=this@entry=0x7f4e48007a60, thread_id=thread_id@entry=0) at util/threadpool_imp.cc:237

19 0x00007f4e5c11729d in rocksdb::ThreadPoolImpl::Impl::BGThreadWrapper (arg=0x7f4e48009970) at util/threadpool_imp.cc:261

20 0x00007f4e5bacf360 in ?? () from /lib64/libstdc++.so.6

21 0x00007f4e5c4b7e25 in start_thread () from /lib64/libpthread.so.0

22 0x00007f4e5ac15bad in clone () from /lib64/libc.so.6

(gdb) f 9

9 rocksdb::CompactionPicker::GetRange (this=0x7f4da0a288c0, inputs=..., smallest=smallest@entry=0x7f4e517f8660, largest=largest@entry=0x7f4e517f8670) at db/compaction_picker.cc:152

152 db/compaction_picker.cc: No such file or directory. (gdb) p inputs $1 = (const rocksdb::CompactionInputFiles &) @0x7f4e517f8760: {level = 2, files = std::vector of length 0, capacity 1} (gdb)

很显然是上面最后一个inputs的vector为空导致的,请fix下这个bug。

absolute8511 commented 6 years ago

能不能提供一下crash前后的一些服务端log. 看官方有个issue没解决, 和这个可能有关

https://github.com/facebook/rocksdb/issues/2560 #