supabase / vecs

Postgres/pgvector Python Client
https://supabase.github.io/vecs/latest
Apache License 2.0
219 stars 33 forks source link

indexes on other tables disappear after i create a new one #90

Closed legaltextai closed 2 months ago

legaltextai commented 2 months ago

I create a new index with docs.create_index() on a new table, and noticed that old indices created similarly on another table disapear. Should I somehow mark all of them as unique? I thought the name of the index gets created with unique name and should not be viewed as duplicate? Thanks!

olirice commented 2 months ago

could you please provide some sample code I can use as a reproduction case?

legaltextai commented 2 months ago

not sure there is much code to provide, other than the vecs code but basically , i create a table

import vecs 
DB_CONNECTION = "__"

vx = vecs.create_client(DB_CONNECTION)

docs = vx.get_or_create_collection(name="adlumal/auslaw-embed-v1.0", dimension=384) 

upload a table id/vec/metadata

then run docs.create_index()

after i run docs.create_index() on another existing table, the one from the first one disappear. my gut feeling is that "is_unique" should be set TRUE by default but I am not 100% certain

i mentioned before that i spent a week recently building an hnsw index after the previous one disappeared somewhere, only now i see what's happening after having played with smaller tables.

to replicate , try to create a couple of tables first and then run create_index over one after another.

if create_index is the same as pgvectors hnsw , cosine , with m=16, and ef_construction = 64, and no more variables, should i run rather pgvector index with these parameters?

legaltextai commented 2 months ago

thanks for addressing this. can i run pip update now ?

olirice commented 2 months ago

thanks for reporting

yes, I just released it in version 0.4.4

https://pypi.org/project/vecs/0.4.4/