Debian support

[enasalitis]

Hi Alex,

Thank you for your great work on bringing vectors to sqlite world. I tried to use sqlite-lembed on a fresh Debian 12 (AMD64) installation (from the official Debian ISO, running on a VMWARE Esxi VM) following the instructions on your blog: https://alexgarcia.xyz/blog/2024/sqlite-lembed-init/index.html

Unfortunately, the sqlite3 command: .load ./lembed0 fails with the following error: Error: ./lembed0.so: undefined symbol: _ZTVN10cxxabiv117class_type_infoE

So it seems that lembed0.so library is not compiled for Debian, so I tried building it myself with no success as it has various other dependencies, most important of which is llama.cpp

Wanted to ask what Linux distro you used for your tests? Would it be possible to publish instructions for building the lembed0.so library on any Linux distro?

[asg017]

Hey @enasalitis apologies for the issues - not sure why the default build doesn't work in debian.

To build yourself, try:

git clone --recurse-submodules https://github.com/asg017/sqlite-lembed

cd sqlite-lembed
./scripts/vendor.sh
make loadable

That should be a dist/lembed0.so extension that you can directly load. Let me know if that works for you

[enasalitis]

Compiled successfully with above instructions but running the sqlite3 load extension from the dist subdirectory: .load ./lembed0 , fails with exactly the same error Error: ./lembed0.so: undefined symbol: _ZTVN10cxxabiv117class_type_infoE

Error in library is confirmed with ldd command: ldd -r lembed0.so _linux-vdso.so.1 (0x00007fff183cd000) libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f0c55a61000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f0c5521f000) /lib64/ld-linux-x86-64.so.2 (0x00007f0c55a89000) undefined symbol: _ZTVN10cxxabiv117class_type_infoE (./lembed0.so) undefined symbol: _ZTVN10cxxabiv120si_class_type_infoE (./lembed0.so) undefined symbol: _ZTINSt6thread6_StateE (./lembed0.so) undefined symbol: _ZTINSt13future_base12_Result_baseE (./lembed0.so) undefined symbol: _ZNKSt19codecvt_utf8_baseIwE6do_outER11mbstate_tPKwS4_RS4_PcS6RS6 (./lembed0.so) undefined symbol: _ZNKSt19codecvt_utf8_baseIwE10do_unshiftER11mbstate_tPcS3RS3 (./lembed0.so) undefined symbol: _ZNKSt19codecvt_utf8_baseIwE5do_inER11mbstate_tPKcS4_RS4_PwS6RS6 (./lembed0.so) undefined symbol: _ZNKSt19codecvt_utf8_baseIwE11do_encodingEv (./lembed0.so) undefined symbol: _ZNKSt19codecvt_utf8_baseIwE16do_always_noconvEv (./lembed0.so) undefined symbol: _ZNKSt19__codecvt_utf8_baseIwE9do_lengthER11mbstate_tPKcS4_m(./lembed0.so) undefined symbol: _ZNKSt19codecvt_utf8_baseIwE13do_max_lengthEv (./lembed0.so) undefined symbol: _ZTISt19codecvt_utf8_baseIwE (./lembed0.so) undefined symbol: _ZTVNSt6thread6_StateE (./lembed0.so) undefined symbol: _ZTISt5ctypeIwE (./lembed0.so) undefined symbol: _ZTISt12future_error (./lembed0.so) undefined symbol: _ZNSt16invalid_argumentD1Ev (./lembed0.so) undefined symbol: _ZSt15once_callable (./lembed0.so) undefined symbol: _ZNSt11regex_errorD1Ev (./lembed0.so) undefined symbol: _ZTVSt19codecvt_utf8_baseIwE (./lembed0.so) undefined symbol: _ZTISt5ctypeIcE (./lembed0.so) undefined symbol: _ZSt11once_call (./lembed0.so) undefined symbol: _ZTVSt12future_error (./lembed0.so) undefined symbol: _ZNSt5ctypeIwE2idE (./lembed0.so) undefined symbol: _ZTISt11regex_error (./lembed0.so) undefined symbol: _ZTISt13runtime_error (./lembed0.so) undefined symbol: _ZTINSt6locale5facetE (./lembed0.so) undefined symbol: _ZNSt6thread4joinEv (./lembed0.so) undefined symbol: _ZNSt13runtime_errorD1Ev (./lembed0.so) undefined symbol: __once_proxy (./lembed0.so) undefined symbol: _ZTISt16invalid_argument (./lembed0.so) undefined symbol: _ZNSt5ctypeIcE2idE (./lembed0.so) undefined symbol: _ZTVSt11regex_error (./lembed0.so) undefined symbol: _ZTIN10cxxabiv115forced_unwindE (./lembed0.so) undefined symbol: _ZTISt12out_of_range (./lembed0.so) undefined symbol: _ZTISt12system_error (./lembed0.so) undefined symbol: _ZTISt9exception (./lembed0.so) undefined symbol: __gxx_personality_v0 (./lembed0.so) undefined symbol: _ZNSt15exception_ptr13exception_ptr10_M_releaseEv (./lembed0.so) undefined symbol: _ZSt20throw_future_errori (./lembed0.so) undefined symbol: _ZSt9use_facetISt5ctypeIwEERKT_RKSt6locale (./lembed0.so) undefined symbol: _ZNSt14basic_ofstreamIcSt11chartraitsIcEEaSEOS2 (./lembed0.so) undefined symbol: _ZSt9use_facetISt5ctypeIcEERKT_RKSt6locale (./lembed0.so) undefined symbol: sqrt (./lembed0.so) undefined symbol: cxa_free_exception (./lembed0.so) undefined symbol: cxa_begin_catch (./lembed0.so) undefined symbol: _ZNKSt9basic_iosIcSt11char_traitsIcEE4failEv (./lembed0.so) undefined symbol: _ZSt28_Rb_tree_rebalance_for_erasePSt18_Rb_tree_nodebaseRS (./lembed0.so) undefined symbol: _ZNSaIcED2Ev (./lembed0.so) undefined symbol: _ZSt24throw_out_of_range_fmtPKcz (./lembed0.so) undefined symbol: floor (./lembed0.so) undefined symbol: ZNSt6localeC1ERKS (./lembed0.so) undefined symbol: _ZdaPv (./lembed0.so) undefined symbol: _ZNSt7cxx1119basic_istringstreamIcSt11char_traitsIcESaIcEED1Ev (./lembed0.so) undefined symbol: _ZNSt7codecvtIwc11mbstate_tEC2Em (./lembed0.so) undefined symbol: _ZNSt13future_base12_Result_baseC2Ev (./lembed0.so) undefined symbol: _ZNSt7__cxx1118basic_stringstreamIcSt11char_traitsIcESaIcEEC1Ev (./lembed0.so) undefined symbol: _ZNSt12future_errorD1Ev (./lembed0.so) undefined symbol: _Znwm (./lembed0.so) undefined symbol: ZNSaIcEC2ERKS (./lembed0.so) undefined symbol: sinf (./lembed0.so) undefined symbol: _ZNSirsERl (./lembed0.so) undefined symbol: _ZNSt7cxx1118basic_stringstreamIcSt11char_traitsIcESaIcEED1Ev (./lembed0.so) undefined symbol: _ZStlsISt11char_traitsIcEERSt13basic_ostreamIcT_ES5_PKc (./lembed0.so) undefined symbol: ZNSaIwEC1ERKS (./lembed0.so) undefined symbol: _ZNSaIwED2Ev (./lembed0.so) undefined symbol: _ZSt19throw_regex_errorNSt15regex_constants10error_typeE (./lembed0.so) undefined symbol: _ZSt20__throw_length_errorPKc (./lembed0.so) undefined symbol: _ZNSt9basic_iosIcSt11char_traitsIcEE4fillEc (./lembed0.so) undefined symbol: cxa_guard_release (./lembed0.so) undefined symbol: _ZNSaIwED1Ev (./lembed0.so) undefined symbol: _ZNSaIwEC2Ev (./lembed0.so) undefined symbol: _ZNSt14basic_ofstreamIcSt11char_traitsIcEEC1ERKNSt7cxx1112basic_stringIcS1_SaIcEEESt13_Ios_Openmode (./lembed0.so) undefined symbol: _ZdlPv (./lembed0.so) undefined symbol: _ZNSt11logicerrorC2ERKS (./lembed0.so) undefined symbol: _ZNSt16invalid_argumentC1EPKc (./lembed0.so) undefined symbol: _ZSt15future_categoryv (./lembed0.so) undefined symbol: _ZNSaIcEC1Ev (./lembed0.so) undefined symbol: _ZNKSt8detail20_Prime_rehash_policy11_M_next_bktEm (./lembed0.so) undefined symbol: _ZSt16throw_bad_castv (./lembed0.so) undefined symbol: powf (./lembed0.so) undefined symbol: _ZNKSt6locale2id5_M_idEv (./lembed0.so) undefined symbol: _ZSt17rethrow_exceptionNSt15exception_ptr13exception_ptrE (./lembed0.so) undefined symbol: cxa_call_unexpected (./lembed0.so) undefined symbol: _ZNSt6localeC1Ev (./lembed0.so) undefined symbol: _ZNSaIwEC1Ev (./lembed0.so) undefined symbol: sqrtf (./lembed0.so) undefined symbol: _ZNSt9basic_iosIcSt11char_traitsIcEE10exceptionsESt12_Ios_Iostate (./lembed0.so) undefined symbol: _Znam (./lembed0.so) undefined symbol: _ZNKSt9basic_iosIwSt11char_traitsIwEE4failEv (./lembed0.so) undefined symbol: fminf (./lembed0.so) undefined symbol: _ZNSt13basic_istreamIwSt11char_traitsIwEErsERl (./lembed0.so) undefined symbol: __cxa_bad_cast (./lembed0.so) undefined symbol: ZNSaIcEC1ERKS (./lembed0.so) undefined symbol: _ZNKSt7cxx117collateIwE9transformEPKwS3_ (./lembed0.so) undefined symbol: _ZStlsIcSt11char_traitsIcESaIcEERSt13basic_ostreamIT_T0_ES7_RKNSt7cxx1112basic_stringIS4_S5_T1_EE (./lembed0.so) undefined symbol: cxa_allocate_exception (./lembed0.so) undefined symbol: _ZNSt7cxx1119basic_istringstreamIwSt11char_traitsIwESaIwEED1Ev (./lembed0.so) undefined symbol: expf (./lembed0.so) undefined symbol: _ZSt20throw_out_of_rangePKc (./lembed0.so) undefined symbol: _ZNSt6thread20hardware_concurrencyEv (./lembed0.so) undefined symbol: _ZNKSt5ctypeIcE13_M_widen_initEv (./lembed0.so) undefined symbol: _ZNSt19codecvt_utf8_baseIwED2Ev (./lembed0.so) undefined symbol: _ZSt29_Rb_tree_insert_and_rebalancebPSt18_Rb_tree_node_baseS0RS (./lembed0.so) undefined symbol: _ZNSt14basic_ofstreamIcSt11char_traitsIcEE5closeEv (./lembed0.so) undefined symbol: _ZNSt14basic_ofstreamIcSt11char_traitsIcEED1Ev (./lembed0.so) undefined symbol: _ZNSt28atomic_futex_unsigned_base19_M_futex_wait_untilEPjjbNSt6chrono8durationIlSt5ratioILl1ELl1EEEENS2_IlS3_ILl1ELl1000000000EEEE (./lembed0.so) undefined symbol: _ZNSirsEPFRSt8ios_baseS0_E (./lembed0.so) undefined symbol: _ZSt9use_facetINSt7cxx117collateIcEEERKT_RKSt6locale (./lembed0.so) undefined symbol: ceilf (./lembed0.so) undefined symbol: _ZSt9use_facetINSt7cxx117collateIwEEERKT_RKSt6locale (./lembed0.so) undefined symbol: _ZNKSt9basic_iosIcSt11char_traitsIcEE5widenEc (./lembed0.so) undefined symbol: cxa_throw (./lembed0.so) undefined symbol: _ZSt17current_exceptionv (./lembed0.so) undefined symbol: cxa_guard_abort (./lembed0.so) undefined symbol: _ZSt17throw_bad_allocv (./lembed0.so) undefined symbol: fmaxf (./lembed0.so) undefined symbol: _ZNKSt7__cxx1119basic_ostringstreamIcSt11char_traitsIcESaIcEE3strEv (./lembed0.so) undefined symbol: _ZNSo5seekpESt4fposI11mbstate_tE (./lembed0.so) undefined symbol: log1pf (./lembed0.so) undefined symbol: ZNSt6localeaSERKS (./lembed0.so) undefined symbol: _ZSt18_Rb_tree_incrementPKSt18_Rb_tree_node_base (./lembed0.so) undefined symbol: _ZNSt15exception_ptr13exception_ptr9_M_addrefEv (./lembed0.so) undefined symbol: _ZNSt7cxx1119basic_istringstreamIcSt11char_traitsIcESaIcEEC1ERKNS_12basic_stringIcS2_S3_EESt13_Ios_Openmode (./lembed0.so) undefined symbol: log2 (./lembed0.so) undefined symbol: cxa_end_catch (./lembed0.so) undefined symbol: __cxa_guard_acquire (./lembed0.so) undefined symbol: _ZNSt7cxx1119basic_ostringstreamIcSt11char_traitsIcESaIcEEC1Ev (./lembed0.so) undefined symbol: _ZNSt13future_base12_Result_baseD2Ev (./lembed0.so) undefined symbol: _ZNSt13runtime_errorC2EPKc (./lembed0.so) undefined symbol: _ZNSt14basic_ofstreamIcSt11char_traitsIcEE7is_openEv (./lembed0.so) undefined symbol: _ZNKSt8detail20_Prime_rehash_policy14_M_need_rehashEmmm (./lembed0.so) undefined symbol: _ZNSt13runtime_errorC1ERKNSt7cxx1112basic_stringIcSt11char_traitsIcESaIcEEE (./lembed0.so) undefined symbol: logf (./lembed0.so) undefined symbol: _ZNSt6localeD1Ev (./lembed0.so) undefined symbol: __cxa_init_primary_exception (./lembed0.so) undefined symbol: _ZNKSt9basic_iosIcSt11char_traitsIcEE4fillEv (./lembed0.so) undefined symbol: floorf (./lembed0.so) undefined symbol: cosf (./lembed0.so) undefined symbol: ZNSaIwEC2ERKS (./lembed0.so) undefined symbol: _ZNSt15exception_ptr13exception_ptrC1EPv (./lembed0.so) undefined symbol: _ZNSt13runtime_errorC1EPKc (./lembed0.so) undefined symbol: _ZNKSt7cxx1118basic_stringstreamIcSt11char_traitsIcESaIcEE3strEv (./lembed0.so) undefined symbol: _ZNSt7cxx1119basic_ostringstreamIcSt11char_traitsIcESaIcEED1Ev (./lembed0.so) undefined symbol: dynamic_cast (./lembed0.so) undefined symbol: _ZSt19throw_logic_errorPKc (./lembed0.so) undefined symbol: _ZNSt6thread6_StateD2Ev (./lembed0.so) undefined symbol: _ZNSo5writeEPKcl (./lembed0.so) undefined symbol: _ZSt18_Rb_tree_incrementPSt18_Rb_tree_node_base (./lembed0.so) undefined symbol: _ZSt25throw_bad_function_callv (./lembed0.so) undefined symbol: _ZStlsISt11char_traitsIcEERSt13basic_ostreamIcT_ES5_c (./lembed0.so) undefined symbol: _ZSt28throw_bad_array_new_lengthv (./lembed0.so) undefined symbol: exp (./lembed0.so) undefined symbol: _ZSt18_Rb_tree_decrementPSt18_Rb_tree_node_base (./lembed0.so) undefined symbol: _ZNSaIcED1Ev (./lembed0.so) undefined symbol: _ZNSaIcEC2Ev (./lembed0.so) undefined symbol: _ZNSt6thread15_M_start_threadESt10unique_ptrINS_6_StateESt14default_deleteIS1_EEPFvvE (./lembed0.so) undefined symbol: _ZNSt3_V216generic_categoryEv (./lembed0.so) undefined symbol: tanhf (./lembed0.so) undefined symbol: _ZSt19throw_range_errorPKc (./lembed0.so) undefined symbol: _ZSt21__glibcxx_assert_failPKciS0S0 (./lembed0.so) undefined symbol: _ZNSt28atomic_futex_unsigned_base19_M_futex_notify_allEPj (./lembed0.so) undefined symbol: _ZNSirsERm (./lembed0.so) undefined symbol: _ZNSt14basic_ofstreamIcSt11char_traitsIcEEC1Ev (./lembed0.so) undefined symbol: _ZNSt11logic_errorC2ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE (./lembed0.so) undefined symbol: _ZNSolsEm (./lembed0.so) undefined symbol: cxa_rethrow (./lembed0.so) undefined symbol: _ZNKSt7cxx117collateIcE9transformEPKcS3_ (./lembed0.so) undefined symbol: log2f (./lembed0.so) undefined symbol: _ZSt20throw_system_errori (./lembed0.so) undefined symbol: roundf (./lembed0.so) undefined symbol: _ZNSt13basic_istreamIwSt11char_traitsIwEErsEPFRSt8ios_baseS4_E(./lembed0.so) undefined symbol: _ZSt9terminatev (./lembed0.so) undefined symbol: _ZSt11_Hash_bytesPKvmm (./lembed0.so) undefined symbol: _ZNSt7cxx1119basic_istringstreamIwSt11char_traitsIwESaIwEEC1ERKNS_12basic_stringIwS2_S3_EESt13_IosOpenmode (./lembed0.so)

Also tried to change Makefile and replace compiler from gcc to g++ , according to the instructions at https://stackoverflow.com/questions/47890484/undefined-symbol-during-dlopen without success.

Any other ideas?