asg017 / sqlite-vss

A SQLite extension for efficient vector search, based on Faiss!
MIT License
1.59k stars 59 forks source link

`UPSERT`-like statements fail on reconstruction #26

Open asg017 opened 1 year ago

asg017 commented 1 year ago

Save this in bug.sql:

.bail on
.mode box
.header on

.load ./vector0
.load ./vss0

create virtual table demo using vss0(a(2));
insert into demo(rowid, a)
  values (1, '[1.0, 2.0]'), (2, '[2.0, 3.0]');

select rowid, vector_debug(a) from demo; -- succeeds

delete from demo where rowid = 1;
insert into demo(rowid, a) select 1, '[99.0, 99.0]';

delete from demo where rowid = 2;
insert into demo(rowid, a) select 2, '[299.0, 299.0]';

select rowid, vector_debug(a) from demo;

And run it:

sqlite3 :memory: '.read bug.sql'
┌───────┬──────────────────────────────┐
│ rowid │       vector_debug(a)        │
├───────┼──────────────────────────────┤
│ 1     │ size: 2 [1.000000, 2.000000] │
│ 2     │ size: 2 [2.000000, 3.000000] │
└───────┴──────────────────────────────┘
Runtime error near line 21: Error reconstructing vector - Does the column factory string end with IDMap2? Full error: Error in virtual void faiss::IndexIDMap2Template<faiss::Index>::reconstruct(idx_t, typename IndexT::component_t *) const [IndexT = faiss::Index] at /Users/runner/work/sqlite-vss/sqlite-vss/vendor/faiss/faiss/IndexIDMap.cpp:236: key 1 not found

Weirdly, if you only "upsert" one row it seems to work just fine...