Extension won't load under Ubuntu

[Adriatic1]

I've built the extension with "make loadable" (added latest sqlite in "vendor" folder), but it fails to load, error being: $ ./sqlite aaa SQLite version 3.45.3 2024-04-15 13:34:05 Enter ".help" for usage hints. sqlite> .load ./vec0 Error: ./vec0.so: undefined symbol: sqrtf

Solution was to link with math lib, i.e. modify Makefile to add "-lm" part at the end of this target:

$(TARGET_LOADABLE): sqlite-vec.c sqlite-vec.h $(prefix)
    gcc \
        -fPIC -shared \
        -Wall -Wextra \
        -Ivendor/ \
        -O3 \
        $(CFLAGS) \
        $< -o $@ \
        -lm

With this the extension will load successfully.

Note that sqlite will crash when trying to execute SQL (below is in gdb with both sqlite and extension compiled with -g): sqlite> create virtual table vec_examples using vec0(sample_embedding float[8]);

Program received signal SIGSEGV, Segmentation fault. 0x0000000000000000 in ?? () (gdb) bt full

0 0x0000000000000000 in ?? ()

No symbol table info available. Backtrace stopped: Cannot access memory at address 0x7ffffffeb928

No luck figuring this out yet, I guess that's material for the next issue. Thanks for working on this, this has great potential.

[asg017]

Thanks for the bug report! Will add -lm to the Makefile to fix the first issue.

For the 2nd issue: Which version of SQLite is that CLI? Does select vec_debug() also cause a crash? Can you also share the output of PRAGMA compile_options; on that CLI?

Also, can you try the linux binaries available in this release? I have a linux github action runner that seems to run/compile the extension fine, so curious to see if that works out-of-the-box for you. You may have to add chmod u+x the downloaded file, forgot to set up permissions correctly.

[Adriatic1]

I've downloaded current latest sqlite release v3.45.3 (amalgamation edition) and compiled the sources with these configuration flags:

SQLITE_FLAGS= -DSQLITE_CORE -DSQLITE_HAS_CODEC -DSQLITE_OMIT_CAST -DSQLITE_OMIT_DEPRECATED -DSQLITE_OMIT_EXPLAIN -DSQLITE_OMIT_UTF16 -DSQLITE_OMIT_PROGRESS_CALLBACK  -DSQLITE_OMIT_SHARED_CACHE -DSQLITE_OMIT_BUILTIN_TEST -DSQLITE_OMIT_CHECK -DSQLITE_OMIT_COMPOUND_SELECT -DSQLITE_OMIT_CONFLICT_CLAUSE -DSQLITE_OMIT_INCRBLOB -DSQLITE_OMIT_MEMORYDB -DSQLITE_OMIT_TCL_VARIABLE -DSQLITE_OMIT_BLOB_LITERAL -DSQLITE_ENABLE_LOCKING_STYLE=0 -DSQLITE_TRACE_SIZE_LIMIT=1024 -DSQLITE_DQS=0 
ifdef WINDIR
 SQLITE_FLAGS += -DSQLITE_OS_WIN
endif

SELECT vec_debug() works OK, see required data below:

sqlite> select vec_debug();
Version: v0.0.1-alpha.3
Date: 2024-05-03T17:42:48Z+0200
Commit: 05bfe5663f99875272c2d7dd7d3a04151f8a024b
Build flags:
sqlite> PRAGMA compile_options;
ATOMIC_INTRINSICS=1
COMPILER=gcc-11.4.0
DEFAULT_AUTOVACUUM
DEFAULT_CACHE_SIZE=-2000
DEFAULT_FILE_FORMAT=4
DEFAULT_JOURNAL_SIZE_LIMIT=-1
DEFAULT_MMAP_SIZE=0
DEFAULT_PAGE_SIZE=4096
DEFAULT_PCACHE_INITSZ=20
DEFAULT_RECURSIVE_TRIGGERS
DEFAULT_SECTOR_SIZE=4096
DEFAULT_SYNCHRONOUS=2
DEFAULT_WAL_AUTOCHECKPOINT=1000
DEFAULT_WAL_SYNCHRONOUS=2
DEFAULT_WORKER_THREADS=0
DIRECT_OVERFLOW_READ
DQS=0
ENABLE_LOCKING_STYLE=0
MALLOC_SOFT_LIMIT=1024
MAX_ATTACHED=10
MAX_COLUMN=2000
MAX_COMPOUND_SELECT=500
MAX_DEFAULT_PAGE_SIZE=8192
MAX_EXPR_DEPTH=1000
MAX_FUNCTION_ARG=127
MAX_LENGTH=1000000000
MAX_LIKE_PATTERN_LENGTH=50000
MAX_MMAP_SIZE=0x7fff0000
MAX_PAGE_COUNT=0xfffffffe
MAX_PAGE_SIZE=65536
MAX_SQL_LENGTH=1000000000
MAX_TRIGGER_DEPTH=1000
MAX_VARIABLE_NUMBER=32766
MAX_VDBE_OP=250000000
MAX_WORKER_THREADS=8
MUTEX_PTHREADS
OMIT_BLOB_LITERAL
OMIT_CAST
OMIT_CHECK
OMIT_COMPOUND_SELECT
OMIT_CONFLICT_CLAUSE
OMIT_DEPRECATED
OMIT_EXPLAIN
OMIT_INCRBLOB
OMIT_MEMORYDB
OMIT_PROGRESS_CALLBACK
OMIT_SHARED_CACHE
OMIT_TCL_VARIABLE
OMIT_UTF16
SYSTEM_MALLOC
TEMP_STORE=1
THREADSAFE=1
sqlite>

Next I've tried your precompiled linux binary, but it doesn't load due to 1st issue I reported here:

vendor$ tar xvf sqlite-vec-0.0.1-alpha.3-loadable-linux-x86_64.tar.gz
vec0.so
vendor$ ./sqlite aaa
SQLite version 3.45.3 2024-04-15 13:34:05
Enter ".help" for usage hints.
sqlite> .load ./vec0
Error: ./vec0.so: undefined symbol: sqrtf
sqlite>

I hope this helped.

[Adriatic1]

The "create virtual table ..." SQL started to work when I recompiled Sqlite with all OMIT flags removed. I guess it would be nice to document what Sqlite features must be compiled in for this extension to work.

[asg017]

Thanks for sharing! Ya looks like the OMIT_INCRBLOB compile-time option is the culprit, sqlite-vec requires the incremental blob i/o API. I'll add a error message at load-time that'll give a warning instead of segfaulting.