asg017 / sqlite-vss

A SQLite extension for efficient vector search, based on Faiss!
MIT License
1.59k stars 59 forks source link

Can it be used in c++? #23

Open jbeuria opened 1 year ago

jbeuria commented 1 year ago

Hello, I am trying to use this library from a c++ code. Could you please help to do so?

asg017 commented 1 year ago

hey yes, it's possible, but not too easy right now.

If you're ok with using the loadable extensions, download the right ones for your operating system/CPU from a recent release, and you can use the sqlite3_load_extension() C/C++ API to load the sqlite-vss extensions.

However, if you want to statically include sqlite-vss in the C++ project, it's not easily possible right now. I'm actually working on making that easier now: The next release of sqlite-vss will also distribute pre-compiled .a and .h files, so you can easily link sqlite-vss into your C/C++ applications. There will also be a new cmake target for static builds, so you can drop a FetchContent() into your CMakeLists.txt and import sqlite-vss into your cmake project.

I'll probably get that up in the next week or two

jbeuria commented 1 year ago

Thanks @asg017 I shall look forward to the next release.

asg017 commented 1 year ago

Hey @jbeuria ,

v0.1.1-alpha.7 now supports static targets for sqlite-vss.

If you check out that release page, there will be "static" releases (ex sqlite-vss-v0.1.1-alpha.7-static-linux-x86_64.tar.gz) for each supported platform that contain .h header files and pre-compiled .a static library files that you can directly use when compiling your C++ project. Also checkout the examples/c directory for an example demo.c and Makefile that shows how to manually link sqlite-vss and its dependencies.

The CMakeLists.txt file also has new sqlite-vector-static and sqlite-vss-static targets. I'm not the best at cmake, but those targets are what generate the libvss0.a and libvector0.a files, and I think you're able to directly use those in other CMake project, but I'm not too sure how.

Please try these out and let me know how it goes! These are very new and may change in the future, and there's definitely room for improvement. In the short-term, I will likely rename the .a files to libsqlite_vector0.a and libsqlite_vss0.a.

jbeuria commented 1 year ago

Thanks for this update. @asg017

I am working on M2 Apple and downloaded sqlite-vss-v0.1.1-alpha.7-static-macos-aarch64.tar.gz

Although I can compile demo.c (with some warnings), the execution gives following error.

❌ demo.c could not load sqlite3_hello_init: bad parameter or other API misuse

asg017 commented 1 year ago

@jbeuria here are some tips:

I'll update demo.c shortly with these updates. I was able to get it working on my M2 mac mini, but it's the same machine where the sqlite-vss-v0.1.1-alpha.7-static-macos-aarch64.tar.gz was compiled.

Can you share the warnings you saw when compiling demo.c ?