search

facebook / rocksdb

A library that provides an embeddable, persistent key-value store for fast storage.
http://rocksdb.org
GNU General Public License v2.0
28.62k stars 6.32k forks source link

Segfault on v7.9.0 trying to open data created by v6.2.2 #10968

Open vadim-berman opened 1 year ago

vadim-berman commented 1 year ago

We've been using RocksDb for years. There's a (.NET) process using RocksDbSharp to generate RocskDb data. There's a C++ library that reads them. Both from 2018 or so. Both use Snappy compression. The C++ library is compiled under C++ 11.

We're trying to upgrade to newer RocksDb now. In unit tests, the new C++ library can create new files and read them just fine. However, it crashes (segmentation fault) on the first rocksdb::DB::Open call when accessing the existing data. More specifically, the call stack goes to LoadTableHandlers.

Here is the source code, nothing fancy:

    rocksdb::DB* db;
    rocksdb::Options options;
    options.compression = rocksdb::CompressionType::kSnappyCompression;
    options.create_if_missing = false;
    rocksdb::Status status = rocksdb::DB::Open(options, myValidPath, &db);

See the call stack below.

ZwWaitForSingleObject 0x00007ffff97ad144 WaitForSingleObjectEx 0x00007ffff6ef306e pthread_join 0x0000000064945edc _ZNSt6thread4joinEv 0x000000006fce0577 rocksdb::VersionBuilder::Rep::LoadTableHandlers(rocksdb::InternalStats, int, bool, bool, std::shared_ptr const&, unsigned long long) 0x00000000009747d8 rocksdb::VersionBuilder::LoadTableHandlers(rocksdb::InternalStats, int, bool, bool, std::shared_ptr const&, unsigned long long) 0x00000000007e86fb rocksdb::VersionEditHandler::LoadTables(rocksdb::ColumnFamilyData, bool, bool) 0x00000000007ea471 rocksdb::VersionEditHandler::CheckIterationResult(rocksdb::log::Reader const&, rocksdb::Status) 0x00000000007ec8c8 rocksdb::VersionEditHandlerBase::Iterate(rocksdb::log::Reader&, rocksdb::Status) 0x00000000007ebe4c rocksdb::VersionSet::Recover(std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator > const&, bool, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, bool) 0x00000000005bc135 rocksdb::DBImpl::Recover(std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator > const&, bool, bool, bool, unsigned long long, rocksdb::DBImpl::RecoveryContext) 0x0000000000570caf rocksdb::DBImpl::Open(rocksdb::DBOptions const&, std::cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator > const&, std::vector<rocksdb::ColumnFamilyHandle, std::allocator<rocksdb::ColumnFamilyHandle> >*, rocksdb::DB**, bool, bool) 0x00000000005698bb rocksdb::DB::Open(rocksdb::Options const&, std::cxx11::basic_string<char, std::char_traits, std::allocator > const&, rocksdb::DB**) 0x000000000056b88f main TestMain.cpp:131 __tmainCRTStartup 0x00000000004013c7 mainCRTStartup 0x00000000004014fb BaseThreadInitThunk 0x00007ffff8a174b4 RtlUserThreadStart 0x00007ffff97626a1

0x0000000000000000 ### Expected behavior Expected to open the file. ### Actual behavior Segmentation fault crash. ### Steps to reproduce the behavior Try opening this file: [role.zip](https://github.com/facebook/rocksdb/files/10049351/role.zip)
vadim-berman commented 1 year ago

Looks like there's a lead.

When DisableWal(1) call is omitted, everything is fine. Was there a feature that was retired?

The reason we called it was to force creation of SST files for smaller tables.

vadim-berman commented 1 year ago

Well, that's odd.

adding this snippet eliminates the crash:

    options.OptimizeForSmallDb();

why, I wonder.