search

mongodb-partners / mongo-rocks

MongoDB storage integration layer for the Rocks storage engine
401 stars 100 forks source link

compilation error on mongo 3.6.5 #146

Open jraverdy-orange opened 6 years ago

jraverdy-orange commented 6 years ago

Hi,

I encounter the following error when trying to compile mongo 3.6.5 with rocksdb 5.14.2 on ubuntu 14.04:

src/mongo/db/modules/rocks/src/rocks_init.cpp: In member function 'virtual mongo::StorageEngine* mongo::{anonymous}::RocksFactory::create(const mongo::StorageGlobalParams&, const mongo::StorageEngineLockFile*) const':
src/mongo/db/modules/rocks/src/rocks_init.cpp:63:62: error: invalid new-expression of abstract class type 'mongo::RocksEngine'                                                     
                                               params.readOnly);
                                                              ^
In file included from src/mongo/db/modules/rocks/src/rocks_init.cpp:39:0:
src/mongo/db/modules/rocks/src/rocks_engine.h:73:11: note:   because the following virtual functions are pure within 'mongo::RocksEngine':                                         
     class RocksEngine final : public KVEngine {
           ^
In file included from src/mongo/db/modules/rocks/src/rocks_engine.h:46:0,
                 from src/mongo/db/modules/rocks/src/rocks_init.cpp:39:
src/mongo/db/storage/kv/kv_engine.h:274:23: note:       virtual mongo::Timestamp mongo::KVEngine::getAllCommittedTimestamp(mongo::OperationContext*) const                         
     virtual Timestamp getAllCommittedTimestamp(OperationContext* opCtx) const = 0;
                       ^
scons: *** [build/opt/mongo/db/modules/rocks/src/rocks_init.o] Error 1
scons: building terminated because of errors.
build/opt/mongo/db/modules/rocks/src/rocks_init.o failed: Error 1

I used the following command to proceed:

  scons -j 4 --ssl --prefix=/opt/mongo   \
  --disable-warnings-as-errors \
  MONGO_VERSION=3.6.5 \
  core install

note that using the same environment/command with mongo 3.4.X, i never had any problem

milkie commented 6 years ago

Which branch of mongo-rocks are you using?

jraverdy-orange commented 6 years ago

i don t checkout from any branch but directly used the tarball from r3.6.5 of mongorocks https://github.com/mongodb-partners/mongo-rocks/archive/r3.6.5.tar.gz

milkie commented 6 years ago

I now see that SERVER-33743 was backported to 3.6.5 and broke compilation for mongo-rocks. Please use 3.6.4 until this can be fixed. @igorcanadi this can probably be fixed by backporting something from the 4.0 branch here?

jraverdy-orange commented 6 years ago

Compilation with r3.6.4 works fine

igorcanadi commented 6 years ago

Yeah, this is a result from https://github.com/mongodb/mongo/commit/2ee6908f1c73dd50d6425e3462ccac2582deb2f3, but that diff hasn't been applied to MongoRocks 4.0 either, so a simple backport is not an option unfortunately.

On Wed, Jul 18, 2018 at 8:54 AM Jérémy RAVERDY notifications@github.com wrote:

Compilation with r3.6.4 works fine

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/mongodb-partners/mongo-rocks/issues/146#issuecomment-405981480, or mute the thread https://github.com/notifications/unsubscribe-auth/ABClzwBUCccBNLWkgyuxW_wwUq65Syusks5uH1oYgaJpZM4VUaZQ .

jraverdy-orange commented 5 years ago

is this issue resolved ?

elirevach12 commented 5 years ago

Hi , I was able to Succesfuly complete 3.6.4 with rock version 5.14.2 , however, when i start the mongod with rocks as storage(WriteTiger is fine ) I getting Segmentation fault error, below root@OPS-02-EUW-MongoBuild1:/compile/mongo# ./mongod_5.14.2 --dbpath /tmp/r --storageEngine rocksdb 2019-01-02T13:37:08.363+0000 I STORAGE [main] Compression: snappy 2019-01-02T13:37:08.363+0000 I STORAGE [main] MaxWriteMBPerSec: 1024 2019-01-02T13:37:08.363+0000 I STORAGE [main] Crash safe counters: 0 2019-01-02T13:37:08.363+0000 I STORAGE [main] Counters: 1 2019-01-02T13:37:08.363+0000 I STORAGE [main] Use SingleDelete in index: 0 2019-01-02T13:37:08.376+0000 I CONTROL [initandlisten] MongoDB starting : pid=96723 port=27017 dbpath=/tmp/r 64-bit host=OPS-02-EUW-MongoBuild1 2019-01-02T13:37:08.376+0000 I CONTROL [initandlisten] db version v3.6.4 2019-01-02T13:37:08.376+0000 I CONTROL [initandlisten] git version: 5e7793fea795097151c0616275e3d3d72e547e7a 2019-01-02T13:37:08.376+0000 I CONTROL [initandlisten] allocator: jemalloc 2019-01-02T13:37:08.376+0000 I CONTROL [initandlisten] modules: rocks 2019-01-02T13:37:08.376+0000 I CONTROL [initandlisten] build environment: 2019-01-02T13:37:08.376+0000 I CONTROL [initandlisten] distarch: x86_64 2019-01-02T13:37:08.376+0000 I CONTROL [initandlisten] target_arch: x86_64 2019-01-02T13:37:08.376+0000 I CONTROL [initandlisten] options: { storage: { dbPath: "/tmp/r", engine: "rocksdb" } } 2019-01-02T13:37:08.376+0000 F - [initandlisten] Invalid access at address: 0x55e527be9b08 2019-01-02T13:37:08.421+0000 F - [initandlisten] Got signal: 11 (Segmentation fault).

0x55e5278952c2 0x55e5278946b9 0x55e527894c97 0x7f5dd9808330 0x55e526097076 0x55e5262a91df 0x55e5262ab133 0x55e5262791dd 0x55e5262e1cbb 0x55e525ee8975 0x55e526092e35 0x55e525f55389 0x7f5dd9450f45 0x55e526091475 ----- BEGIN BACKTRACE ----- {"backtrace":[{"b":"55E525567000","o":"232E2C2","s":"_ZN5mongo15printStackTraceERSo"},{"b":"55E525567000","o":"232D6B9"},{"b":"55E525567000","o":"232DC97"},{"b":"7F5DD97F8000","o":"10330"},{"b":"55E525567000","o":"B30076","s":"_ZNSt16_Sp_counted_baseILN9gnu_cxx12_Lock_policyE2EE10_M_releaseEv"},{"b":"55E525567000","o":"D421DF","s":"_ZNK5mongo11RocksEngine8_optionsEv"},{"b":"55E525567000","o":"D44133","s":"_ZN5mongo11RocksEngineC2ERKSsbib"},{"b":"55E525567000","o":"D121DD"},{"b":"55E525567000","o":"D7ACBB","s":"_ZN5mongo20ServiceContextMongoD29initializeGlobalStorageEngineEv"},{"b":"55E525567000","o":"981975"},{"b":"55E525567000","o":"B2BE35","s":"ZN5mongo11mongoDbMainEiPPcS1"},{"b":"55E525567000","o":"9EE389","s":"main"},{"b":"7F5DD942F000","o":"21F45","s":"__libc_start_main"},{"b":"55E525567000","o":"B2A475"}],"processInfo":{ "mongodbVersion" : "3.6.4", "gitVersion" : "5e7793fea795097151c0616275e3d3d72e547e7a", "compiledModules" : [ "rocks" ], "uname" : { "sysname" : "Linux", "release" : "4.4.0-97-generic", "version" : "#120~14.04.1-Ubuntu SMP Wed Sep 20 15:53:13 UTC 2017", "machine" : "x86_64" }, "somap" : [ { "b" : "55E525567000", "elfType" : 3, "buildId" : "233950F62EF8331963FEF56AC6C20E9D740D52A7" }, { "b" : "7FFE6FBF6000", "elfType" : 3, "buildId" : "031F7BA6B1BBA04496497C5611C43E0EBE4CE00D" }, { "b" : "7F5DDA343000", "path" : "/lib/x86_64-linux-gnu/libbz2.so.1.0", "elfType" : 3, "buildId" : "E1031DDBFFE20367E874B7093EEC0C8D9F3B43F6" }, { "b" : "7F5DDA128000", "path" : "/lib/x86_64-linux-gnu/libresolv.so.2", "elfType" : 3, "buildId" : "AD304AFCE6847F7A4D66D22853E87CCBF5A66966" }, { "b" : "7F5DD9F20000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3, "buildId" : "4F930712D3609C93E380E5BE5DF73E7AD273531C" }, { "b" : "7F5DD9D1C000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3, "buildId" : "034D6A4EE9DCAB4A34ABD644345CBBB42DC63088" }, { "b" : "7F5DD9A16000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3, "buildId" : "300C7884CDEB5667BEA2357D2B8E7A76397562D6" }, { "b" : "7F5DD97F8000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3, "buildId" : "F64B8AD471FBA1B7A3A64EFB01551E694975E1F7" }, { "b" : "7F5DD942F000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3, "buildId" : "D9A10B8EF90300628DD0A3A535106967714D7328" }, { "b" : "7F5DDA553000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "2CA513EDC89C7BC06EC183D1A3A03CC0F606319C" } ] }} mongod_5.14.2(_ZN5mongo15printStackTraceERSo+0x32) [0x55e5278952c2] mongod_5.14.2(+0x232D6B9) [0x55e5278946b9] mongod_5.14.2(+0x232DC97) [0x55e527894c97] libpthread.so.0(+0x10330) [0x7f5dd9808330] mongod_5.14.2(_ZNSt16_Sp_counted_baseILN9gnu_cxx12_Lock_policyE2EE10_M_releaseEv+0x16) [0x55e526097076] mongod_5.14.2(_ZNK5mongo11RocksEngine8_optionsEv+0x27F) [0x55e5262a91df] mongod_5.14.2(_ZN5mongo11RocksEngineC2ERKSsbib+0x7F3) [0x55e5262ab133] mongod_5.14.2(+0xD121DD) [0x55e5262791dd] mongod_5.14.2(_ZN5mongo20ServiceContextMongoD29initializeGlobalStorageEngineEv+0x62B) [0x55e5262e1cbb] mongod_5.14.2(+0x981975) [0x55e525ee8975] mongod_5.14.2(ZN5mongo11mongoDbMainEiPPcS1+0xC55) [0x55e526092e35] mongod_5.14.2(main+0x9) [0x55e525f55389] libc.so.6(__libc_start_main+0xF5) [0x7f5dd9450f45] mongod_5.14.2(+0xB2A475) [0x55e526091475] ----- END BACKTRACE -----

elirevach12 commented 5 years ago

I use this command to build the rocksdb part CFLAGS="-march=core-avx-i -fPIC" JEMALLOC=1 PORTABLE=1 USE_RTTI=1 make V=1 -j 16 static_lib

elirevach12 commented 5 years ago

Output of the core file: rogram terminated with signal SIGSEGV, Segmentation fault.

0 std::_Sp_counted_base<(gnu_cxx::_Lock_policy)2>::_M_release (this=0x7f58695c3500 <_GIclock_gettime>) at /usr/include/c++/5/bits/shared_ptr_base.h:147

147 if (__gnu_cxx::__exchange_and_add_dispatch(&_M_use_count, -1) == 1) [Current thread is 1 (Thread 0x7f586a7edcc0 (LWP 69178))] MongoDB GDB commands loaded, run 'mongodb-help' for list of commands Warning: Could not load bson library for Python '3.4.3 (default, Nov 12 2018, 22:32:28) [GCC 4.8.4]'. --Type for more, q to quit, c to continue without paging--where Check with the pip command if pymongo 3.x is installed. MongoDB GDB pretty-printers loaded MongoDB Lock analysis commands loaded (gdb) where

0 std::_Sp_counted_base<(gnu_cxx::_Lock_policy)2>::_M_release (this=0x7f58695c3500 <_GIclock_gettime>) at /usr/include/c++/5/bits/shared_ptr_base.h:147

1 std::shared_count<(__gnu_cxx::_Lock_policy)2>::~shared_count (this=0x7ffcaa06ef98, __in_chrg=) at /usr/include/c++/5/bits/shared_ptr_base.h:659

2 std::shared_ptr<rocksdb::TableFactory, (__gnu_cxx::_Lock_policy)2>::~shared_ptr (this=0x7ffcaa06ef90, __in_chrg=)

at /usr/include/c++/5/bits/shared_ptr_base.h:925

3 std::shared_ptr::~shared_ptr (this=0x7ffcaa06ef90, __in_chrg=) at /usr/include/c++/5/bits/shared_ptr.h:93

4 rocksdb::ColumnFamilyOptions::~ColumnFamilyOptions (this=0x7ffcaa06edc8, __in_chrg=) at /usr/local/include/rocksdb/options.h:81

5 0x00005623745e14f6 in rocksdb::ColumnFamilyDescriptor::~ColumnFamilyDescriptor (this=0x7ffcaa06edc0, __in_chrg=) at ./include/rocksdb/db.h:60

6 rocksdb::DB::Open (options=..., dbname=..., dbptr=dbptr@entry=0x7ffcaa06f100) at db/db_impl_open.cc:990

7 0x0000562372ebf69c in mongo::RocksEngine::RocksEngine (this=0x562376e43340, path=..., durable=, formatVersion=, readOnly=)

at src/mongo/db/modules/rocks/src/rocks_engine.cpp:205

8 0x0000562372e8d88d in mongo::(anonymous namespace)::RocksFactory::create (this=0x562376e40980, params=..., lockFile=)

at src/mongo/db/modules/rocks/src/rocks_init.cpp:63

9 0x0000562372ef59cb in mongo::ServiceContextMongoD::initializeGlobalStorageEngine (this=0x562376a76800) at src/mongo/db/service_context_d.cpp:202

10 0x0000562372aedc07 in mongo::(anonymous namespace)::_initAndListen (listenPort=, this=) at src/mongo/db/db.cpp:742

11 0x0000562372ca8455 in mongo::(anonymous namespace)::initAndListen (listenPort=) at src/mongo/db/db.cpp:1060

12 mongo::mongoDbMain (argc=5, argv=0x7ffcaa070ee8, envp=) at src/mongo/db/db.cpp:1412

13 0x0000562372b5a629 in main (argc=, argv=, envp=) at src/mongo/db/dbmain.cpp:49

(gdb)

y123456yz commented 5 years ago

2019-01-02T13:37:08.376+0000 I CONTROL [initandlisten] target_arch: x86_64 2019-01-02T13:37:08.376+0000 I CONTROL [initandlisten] options: { storage: { dbPath: "/tmp/r", engine: "rocksdb" } } 2019-01-02T13:37:08.376+0000 F - [initandlisten] Invalid access at address: 0x55e527be9b08

+1, but my mongodb is 3.4.20 mongo-rocks is 3.4

wolfkdy commented 4 years ago

I ought to think mongoRocks3.6 is a skipped version. I guess Perhaps you want to use changeStream? is it possible that you try mongoRocks 4.0?