[Android] Abort message: assertion "terminating with uncaught exception of type std::length_error: vector" failed'

[vinovarkey]

Team,

I have compiled rocksdb for Android using the following cmake command

${ANDROID_HOME}/cmake/3.6.4111459/bin/cmake \
    -DANDROID_ABI=arm64-v8a \
    -DANDROID_NDK=${ANDROID_HOME}/ndk-bundle \
    -DCMAKE_ANDROID_STL_TYPE=c++_shared \
    -DCMAKE_BUILD_TYPE=Debug \
    -DCMAKE_TOOLCHAIN_FILE=${ANDROID_NDK}/build/cmake/android.toolchain.cmake \
    -DCMAKE_LIBRARY_OUTPUT_DIRECTORY=${LIB_DIR} \
    -DCMAKE_ARCHIVE_OUTPUT_DIRECTORY=${LIB_DIR} \
    -DANDROID_NATIVE_API_LEVEL=19 \
    -DANDROID_LINKER_FLAGS="-landroid -llog" \
    -DANDROID_CPP_FEATURES="rtti exceptions" \
    -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON \
    -DPORTABLE:BOOL=ON \
    -DFAIL_ON_WARNINGS:BOOL=OFF \
    -DWITH_TESTS:BOOL=OFF \
    -DWITH_TOOLS:BOOL=OFF \
    -DWITH_GFLAGS:BOOL=OFF
    /path/to/dir/containing/rocksdb/CMainLists.txt

followed by a $ANDROID_CMAKE --build /path/containing/CMakeCache.txt

I have bundled the resulting librocksdb.so into a sample Android app. I have the following JNI code snippet which crashes

using namespace rocksdb;

extern "C" {
JNIEXPORT jint JNICALL Java_sink_kitchen_com_MainActivity_rocks(JNIEnv *env, jobject thiz, jstring path) {
    std::string str = "/storage/emulated/0/Android/data/sink.kitchen.com/files/a.rocksdb";
    DB* db;
    Options options;
    options.create_if_missing = true;
    options.create_missing_column_families = true;
    options.OptimizeForSmallDb();
    Status status = DB::Open(options, str, &db);
    return 1;
}
}

The crash exception trace is

DEBUG: *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** 
DEBUG: Build fingerprint: 'Android/vbox86p/vbox86p:8.0.0/OPR6.170623.017/49:userdebug/test-keys'
DEBUG: Revision: '0'
DEBUG: ABI: 'x86'
DEBUG: pid: 2913, tid: 2913, name: ink.kitchen.com  >>> sink.kitchen.com <<<
DEBUG: signal 6 (SIGABRT), code -6 (SI_TKILL), fault addr --------
DEBUG: Abort message: '/Volumes/Android/buildbot/src/android/ndk-release-r17/external/libcxx/../../external/libcxxabi/src/abort_message.cpp:73: abort_message: assertion "terminating with uncaught exception of type std::length_error: vector" failed'
DEBUG:     eax 00000000  ebx 00000b61  ecx 00000b61  edx 00000006
DEBUG:     esi 00000b61  edi ffa3ab48
DEBUG:     xcs 00000023  xds 0000002b  xes 0000002b  xfs 0000006b  xss 0000002b
DEBUG:     eip f2cffbc9  ebp ffa3ab68  esp ffa3aafc  flags 00000296
DEBUG: 
DEBUG: backtrace:
DEBUG:     #00 pc 00000bc9  [vdso:f2cff000] (__kernel_vsyscall+9)
DEBUG:     #01 pc 00075dcc  /system/lib/libc.so (tgkill+28)
DEBUG:     #02 pc 0001f05e  /system/lib/libc.so (abort+110)
DEBUG:     #03 pc 0002721e  /system/lib/libc.so (__libc_fatal+46)
DEBUG:     #04 pc 0001f4c0  /system/lib/libc.so (__assert2+48)  
DEBUG:     #05 pc 005264a4  /data/app/sink.kitchen.com-5M0CohrwLzsLoUwPrWn2lQ==/lib/x86/libnative.so
DEBUG:     #06 pc 00526617  /data/app/sink.kitchen.com-5M0CohrwLzsLoUwPrWn2lQ==/lib/x86/libnative.so
DEBUG:     #07 pc 00522829  /data/app/sink.kitchen.com-5M0CohrwLzsLoUwPrWn2lQ==/lib/x86/libnative.so
DEBUG:     #08 pc 00521d9e  /data/app/sink.kitchen.com-5M0CohrwLzsLoUwPrWn2lQ==/lib/x86/libnative.so
DEBUG:     #09 pc 00521cf3  /data/app/sink.kitchen.com-5M0CohrwLzsLoUwPrWn2lQ==/lib/x86/libnative.so (__cxa_throw+115)
DEBUG:     #10 pc 0031d493  /data/app/sink.kitchen.com-5M0CohrwLzsLoUwPrWn2lQ==/lib/x86/libnative.so (_ZNKSt6__ndk120__vector_base_commonILb1EE20__throw_length_errorEv+163)
DEBUG:     #11 pc 00326450  /data/app/sink.kitchen.com-5M0CohrwLzsLoUwPrWn2lQ==/lib/x86/libnative.so (_ZNSt6__ndk16vectorIiNS_9allocatorIiEEE8allocateEj+96)
DEBUG:     #12 pc 0031f309  /data/app/sink.kitchen.com-5M0CohrwLzsLoUwPrWn2lQ==/lib/x86/libnative.so (_ZNSt6__ndk16vectorIiNS_9allocatorIiEEEC2ERKS3_+313)
DEBUG:     #13 pc 00325284  /data/app/sink.kitchen.com-5M0CohrwLzsLoUwPrWn2lQ==/lib/x86/libnative.so (_ZN7rocksdb27AdvancedColumnFamilyOptionsC2ERKS0_+340)
DEBUG:     #14 pc 00320246  /data/app/sink.kitchen.com-5M0CohrwLzsLoUwPrWn2lQ==/lib/x86/libnative.so (_ZN7rocksdb19ColumnFamilyOptionsC2ERKS0_+70)
DEBUG:     #15 pc 003201eb  /data/app/sink.kitchen.com-5M0CohrwLzsLoUwPrWn2lQ==/lib/x86/libnative.so (_ZN7rocksdb19ColumnFamilyOptionsC1ERKNS_7OptionsE+107)
DEBUG:     #16 pc 0031a6d4  /data/app/sink.kitchen.com-5M0CohrwLzsLoUwPrWn2lQ==/lib/x86/libnative.so (Java_sink_kitchen_com_MainActivity_rocks+196)
DEBUG:     #17 pc 000090a8  /data/app/sink.kitchen.com-5M0CohrwLzsLoUwPrWn2lQ==/oat/x86/base.odex (offset 0x9000)
DEBUG:     #18 pc 00049fff  [anon:libc_malloc:f2a80000]
DEBUG:     #19 pc 00051c97  /dev/ashmem/dalvik-main space (region space) (deleted)
DEBUG:     #20 pc 80876e11  <unknown>
DEBUG:     #21 pc 89a66073  <unknown>
DEBUG:     #22 pc 0d3fffff  /dev/ashmem/dalvik-main space (region space) (deleted)                  

I have been able to reproduce the crash with a much simpler code than DB::Open(options, str, &db):

using namespace rocksdb;
extern "C" {
JNIEXPORT jint JNICALL Java_sink_kitchen_com_MainActivity_rocks(JNIEnv *env, jobject thiz) {
    Options options;
    options.create_if_missing = true;
    options.create_missing_column_families = true;
    options.OptimizeForSmallDb();
    ColumnFamilyOptions cf_options(options);
    return 1;
}
}

After stepping through the code above, I have noticed the exception occurs when initializing AdvancedColumnFamilyOptions's std::vector<int> max_bytes_for_level_multiplier_additional =std::vector<int>(num_levels, 1);

Here is the screenshot after stepping through the code

Here is the max_size() output from debugger

(lldb) p max_size()
(std::__ndk1::vector<int, std::__ndk1::allocator<int> >::size_type) $2 = 1073741823

And here is the backtrace at this point in time

(lldb) bt
* thread #1, name = 'ink.kitchen.com', stop reason = step over
  * frame #0: 0xd3bb34e1 libnative.so`std::__ndk1::vector<int, std::__ndk1::allocator<int> >::allocate(this=0x00000000ffa3ac74 size=1, __n=3841141408) at vector:925
    frame #1: 0xd3bac3aa libnative.so`std::__ndk1::vector<int, std::__ndk1::allocator<int> >::vector(this=0x00000000ffa3ac74 size=1, __x=size=1) at vector:1201
    frame #2: 0xd3bb2325 libnative.so`rocksdb::AdvancedColumnFamilyOptions::AdvancedColumnFamilyOptions(this=0xffa3ac10, (null)=0xffa3aec4) at advanced_options.h:164
    frame #3: 0xd3bad2e7 libnative.so`rocksdb::ColumnFamilyOptions::ColumnFamilyOptions(this=0xffa3ac10, (null)=0xffa3aec4) at options.h:82
    frame #4: 0xd3bad28c libnative.so`rocksdb::ColumnFamilyOptions::ColumnFamilyOptions(this=0xffa3ac10, options=0xffa3ad90) at options.cc:103
    frame #5: 0xd3ba774f libnative.so`::Java_sink_kitchen_com_MainActivity_rocks(env=0xf2aa9230, thiz=0xffa3b08c, path=0xffa3b090) at native.cpp:30
    frame #6: 0xd410d0a9 base.odex`rocks(this=<unavailable>, (null)=<unavailable>) + 137
    frame #7: 0xee521cd3 libart.so`art_quick_invoke_stub + 339
    frame #8: 0xedffbb49 libart.so`art::ArtMethod::Invoke(art::Thread*, unsigned int*, unsigned int, art::JValue*, char const*) + 233

libnative.so is where the JNI code is bundled into.

As suggested in https://github.com/facebook/rocksdb/issues/2809, I have tried the .so build with and without the ROCKSDB_LITE option. I still run into the same crash.

Any ideas on how I can proceed further?

[sagar0]

What version of RocksDB are you using?

@promethe42 was able to successfully build RocksDB targeting Android as mentioned here: https://github.com/facebook/rocksdb/issues/1193#issuecomment-411318485 , after a bug fix that he did in #4231 .

[JMLX42]

@sagar0 building yes, testing no... at least not yet! I'll report here ASAP.

[suzanne-stathatos]

any update on this issue? were you able to get it to run?

[JMLX42]

Yes. We got it to run without trouble. So I can't reproduce.

[Domos]

I ran into the same issue with ROCKSDB_LITE build. Then I realized my project does not have ROCKSDB_LITE compiler definition. https://github.com/facebook/rocksdb/blob/327c4807a7fe8532326323e2753670daf06a0f6b/include/rocksdb/options.h#L994-L1001 add_definitions() only adds definitions to rocksdb project, not the user projects. Suggest to add target_compile_definitions() in the CMAKE config. https://github.com/facebook/rocksdb/blob/327c4807a7fe8532326323e2753670daf06a0f6b/CMakeLists.txt#L393-L396

[gritzko]

@Domos I almost went insane. Thank you!