Mac M1 load error: dlsym(0x309d14370, sqlite3_vss_init): symbol not found

[cmcilvoy]

M1 builds successfully but unable to load extension into SQLite version 3.36.0 Mac 13.4.1 /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk Source package sqlite-vss-0.1.1-alpha.20

sqlite> .load /Users/user/Desktop/sqlite-vss/sqlite-vss/build_release/vss0.dylib Error: dlsym(0x309d14370, sqlite3_vss_init): symbol not found

[polterguy]

Do you get the same error if you're using the pre-built libraries?

[cmcilvoy]

Yes. Thanks for quick response.

(including .dylib it seems to duplicate) sqlite> .load /Users/user/Downloads/sqlite-vss-v0.1.1-alpha.20-loadable-macos-aarch64/vector0.dylib Error: dlopen(/Users/user/Downloads/sqlite-vss-v0.1.1-alpha.20-loadable-macos-aarch64/vector0.dylib.dylib, 0x000A): tried: '/Users/user/Downloads/sqlite-vss-v0.1.1-alpha.20-loadable-macos-aarch64/vector0.dylib.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/user/Downloads/sqlite-vss-v0.1.1-alpha.20-loadable-macos-aarch64/vector0.dylib.dylib' (no such file), '/U

(removing it seems to not like architecture) sqlite> .load /Users/user/Downloads/sqlite-vss-v0.1.1-alpha.20-loadable-macos-aarch64/vector0 Error: dlopen(/Users/user/Downloads/sqlite-vss-v0.1.1-alpha.20-loadable-macos-aarch64/vector0.dylib, 0x000A): tried: '/Users/user/Downloads/sqlite-vss-v0.1.1-alpha.20-loadable-macos-aarch64/vector0.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/System/Volumes/Preboot/Cryptexes/OS/Users/user/Downloads/sqlite-vss-v0.1.1-alpha.20-loa

[asg017]

Can you run uname -s -m on your machine and share what it prints out?

The mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64') message makes you think you might be on a x86_64 machine.

But regardless of that, if you build it yourself you shouldnt get the 'sqlite3_vss_init` symbol not found error. Can you also run:

nm -g /Users/user/Desktop/sqlite-vss/sqlite-vss/build_release/vss0.dylib | grep sqlite3_

[cmcilvoy]

uname -s -m Darwin arm64

nm -g /Users/cmcilvoy/Desktop/sqlite-vss/sqlite-vss/buildrelease/vss0.dylib | grep sqlite3 /Users/user/Desktop/sqlite-vss/sqlite-vss/build_release/vss0.dylib: no symbols

[asg017]

hm alright that's really weird, a few questions:

• How did you build build_release/vss0.dylib? Did you git clone and run make loadable-release?
• Can you try the binaries at https://github.com/asg017/sqlite-vss/releases/download/v0.1.1-alpha.20/sqlite-vss-v0.1.1-alpha.20-loadable-macos-aarch64.tar.gz ? And also run the nm -g command on those .dylib files?
• Can you run sw_vers?

[cmcilvoy]

• Yes, followed your exact steps and did then loadable-release

• nm -g /Users/cmcilvoy/Downloads/sqlite-vss-v0.1.1-alpha.20-loadable-macos-aarch64\ 2/vector0.dylib 000000000001c000 D _VECTOR_BLOB_HEADER_BYTE 000000000001c001 D _VECTOR_BLOB_HEADER_TYPE 000000000001c008 D _VECTOR_FLOAT_POINTER_NAME U Unwind_Resume 000000000000326c T __Z12resultVectorP15sqlite3_contextPNSt316vectorIfNS1_9allocatorIfEEEE 0000000000003238 T Z14delVectorFloatPv 00000000000032f8 T Z19vectorFromBlobValueP13sqlite3_valuePPKc 000000000000359c T Z19vectorFromTextValueP13sqlite3_value 0000000000003460 T __Z22vectorFromRawBlobValueP13sqlite3_valuePPKc U ZNKSt13runtime_error4whatEv U ZNKSt3119shared_weak_count13get_deleterERKSt9type_info U ZNKSt9exception4whatEv U ZNSt11logic_errorC2EPKc U ZNSt12length_errorD1Ev U ZNSt12out_of_rangeD1Ev U ZNSt13runtime_errorC1EPKc U ZNSt13runtimeerrorC1ERKS U ZNSt13runtime_errorD1Ev U __ZNSt20bad_array_new_lengthC1Ev U ZNSt20bad_array_new_lengthD1Ev U ZNSt3112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE6appendEPKc U ZNSt3112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE6appendEPKcm U ZNSt3112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE6resizeEmc U ZNSt3112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE7reserveEm U ZNSt3112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE9push_backEc U ZNSt3112basic_stringIcNS_11char_traitsIcEENS9allocatorIcEEEC1ERKS5 U ZNSt3119shared_weak_count14release_weakEv U ZNSt3119shared_weak_countD2Ev U ZNSt319to_stringEi U ZNSt319to_stringEm U ZNSt9exceptionD2Ev U ZSt9terminatev U ZTINSt3119shared_weak_countE U ZTISt12length_error U ZTISt12out_of_range U ZTISt20bad_array_new_length U ZTISt9exception U ZTVN10cxxabiv117class_type_infoE U ZTVN10cxxabiv120si_class_type_infoE U ZTVSt12length_error U ZTVSt12out_of_range U ZdlPv U _Znwm U cxa_allocate_exception U _cxa_begincatch U cxa_end_catch U _cxa_freeexception U cxa_throw U error U gxx_personality_v0 U _stack_chkfail U stack_chk_guard U _bzero U _localeconv U _memcmp U _memcpy U _memmove U _memset U _snprintf 000000000001c0d0 S _sqlite3_api 0000000000003a28 T _sqlite3_vector_init U _strlen U _strtod U _strtoll U _strtoull

• sw_vers ProductName: macOS ProductVersion: 13.4.1 BuildVersion: 22F82

[asg017]

Ok so it seems like if you .load the vector0 file inside the Downloads/sqlite-vss-v0.1.1-alpha.20-loadable-macos-aarch64\ 2/ directory, it should work as expected. Be careful with the space (" ") character in that file path when you try, you can cd into that directory first and run sqlite3 with .load ./vector0 .

I suspect that .load ./vss0 in that directory should also work. If it doesn't, could you run nm -g vss0.dylib | grep sqlite3_ on that as well?

Re building yourself: if you did follow those steps then I have no idea why you get no symbols on there. You can also look in the sqlite-vss/dist/release directory, which should also have a vector0.dylib and vss0.dylib files. They're normally the same as the files in build_release, but something might've gotten corrupted

[cmcilvoy]

Ugh same incompatible message. Really weird.

@McLap > ~/Downloads/sqlite-vss-v0.1.1-alpha.20-loadable-macos-aarch64 /usr/local/opt/sqlite/bin/sqlite3 SQLite version 3.36.0 2021-06-18 18:36:39 Enter ".help" for usage hints. Connected to a transient in-memory database. Use ".open FILENAME" to reopen on a persistent database. sqlite> .load vector0 Error: dlopen(vector0.dylib, 0x000A): tried: 'vector0.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/System/Volumes/Preboot/Cryptexes/OSvector0.dylib' (no such file), '/usr/lib/vector0.dylib' (no such file, not in dyld cache), 'vector0.dylib' (mach-o file, but is an i sqlite> .load vector0.dylib Error: dlopen(vector0.dylib.dylib, 0x000A): tried: 'vector0.dylib.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OSvector0.dylib.dylib' (no such file), '/usr/lib/vector0.dylib.dylib' (no such file, not in dyld cache), 'vector0.dylib.dylib' (no such file), '/usr/local/lib/vector0.dylib.dylib' (no such file) sqlite> .load vector0.dylib

@McLap > nm -g /Users/cmcilvoy/Desktop/sqlite-vss/sqlite-vss/dist/release/vector0.dylib /Users/cmcilvoy/Desktop/sqlite-vss/sqlite-vss/dist/release/vector0.dylib: no symbols @McLap > nm -g /Users/cmcilvoy/Desktop/sqlite-vss/sqlite-vss/dist/release/vss0.dylib /Users/cmcilvoy/Desktop/sqlite-vss/sqlite-vss/dist/release/vss0.dylib: no symbols

[asg017]

Can you run file /usr/local/opt/sqlite/bin/sqlite3 ?

On my M1 mac, I installed sqlite3 with homebrew so it returns:

file /opt/homebrew/opt/sqlite/bin/sqlite3
/opt/homebrew/opt/sqlite/bin/sqlite3: Mach-O 64-bit executable arm64

I suspect that the 'vector0.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')) error may be coming from some MacOS Rosetta stuff that's trying to run x86_64 programs on your M1. So I'm thinking /usr/local/opt/sqlite/bin/sqlite3 may be a x86_64 build of sqlite3 and is causing this issue

[cmcilvoy]

That was it. I did a brew reinstall and its working properly. thanks!

[asg017]

Glad to hear! Will add this problem to a "troubleshooting" doc in the near future