search

timescale / pgvectorscale

A complement to pgvector for high performance, cost efficient vector search on large workloads.
PostgreSQL License
1.32k stars 56 forks source link

Setup client for pgvectorscale #128

Open Drzhivago264 opened 2 months ago

Drzhivago264 commented 2 months ago

Hello thank you for the project, Can you please clarify about the process of using this extension with pgvector client. For now I guess the process is: 1) setup pgvector and pgvectorscale. 2) install client and setup a table with embeddingfield (ignore pgvector indexes option) 3) go down to sql server and manually setup the new fancier ANN index things (i read both papers and grasp the idea but this seems like big brain businesses). 4) go up to high level client and do business as usual.

Here is my question: 1) i believe the indexes manage itself so no need to rebuild after a specific period of time. 2) current pgvector clients are compatible with pgvectorscale, meaning there is no raw sql queries differences between the two.

I am ask this because I am using Django and it will be a pain for me to have one table that needs custom raw sql and to be manually managed outside of Django Orm and model management. Thank you for your clarification.

### Tasks
jumasheff commented 1 month ago

Came here to upvote this comment. I am going to use this extension in my Django project as well. I think your project is gaining momentum since people started to realize that you can use battle tested technology stack in a RAG setting without having to use fancy vector DBs.

cevian commented 1 month ago

Your understanding is correct.

  1. You do not need to rebuild the index. Though we do recommend building the index on at least some data (not an empty table). The reason is the SBQ compression is better able to "learn" the data.
  2. The clients are compatible with pgvectorscale. The index used is transparent to the client.

The only difference should indeed be the manual sql index creation step.