search

asg017 / sqlite-vss

A SQLite extension for efficient vector search, based on Faiss!
MIT License
1.71k stars 62 forks source link

Runtime error: vss_search() only support vss_search_params() as a 2nd parameter for SQLite versions below 3.41.0 #25

Closed petegordon closed 1 year ago

petegordon commented 1 year ago

Hey! Thanks for doing this project, it is super cool!

Runtime error: vss_search() only support vss_search_params() as a 2nd parameter for SQLite versions below 3.41.0

I hit this issue doing a simple example like the documentation.

image

Any guidance on getting around it. Other than having to downgrade sqlite, or is that what is recommended? I saw that the issue on the sqlite site had seemed to be addressed. Maybe I should pull the sqlite-vss code and try to build locally? Thanks! https://github.com/asg017/sqlite-vss/blob/589c2cada6f720cce85c8cc80cfd3e0e43600f85/src/vss-extension.cpp#L835 https://sqlite.org/forum/info/6b32f818ba1d97ef

asg017 commented 1 year ago

Hey thanks for filing! That error message isn't great, but I think there's an internal bug with it (it's showing the wrong error message in this statement)

Can you add a LIMIT clause to your query, to something like this?

select rowid, distance 
from vss_section_text 
where vss_search(
  search_embeddings, 
  (select section_embeddings from section_text where rowid = 9)
)
limit 10;

Some background info: the LIMIT clause is enforced on vss_search queries because without it, the entire vector database will be search + sorted which is expensive. Though now that I think about it, I think it causes more confusion that anything, so I'll possibly drop that requirement in a future release

Let me know if the LIMIT doesn't fix your problem!

petegordon commented 1 year ago

That was absolutely it! (And, I had another problem with my dimension size being wrong for what it should have been. But, fixing those two things worked like a champ! Thanks!

image