[Question]: For a scaling application, should you use multiple indices or only one index in a Postgres Vectorstore?

[Falco-Boehnke]

Question Validation

• [X] I have searched both the documentation and discord for an answer.

Question

I have searched the documentation but I can't find any conclusive information on this topic.

As it stands, we are using PGVectorStore with an ingestion pipeline to create embeddings. As I understand, from_vector_store creates an index using those embeddings. After creation of the index, it is stored within the vectorstore and called up during question time.

Right now we use a unique id to distinguish users. For this setup, shuold we use a singular large index, or create multiple indices based on the users? Also, is that even possible with a Postgres Vectorstore? For a redis store for example, it seems only a singular index can be used

[dosubot[bot]]

Hey there, @Falco-Boehnke! I'm here to help you with any bugs, questions, or contributions. Let's tackle this together!

You can create multiple indices based on users in a Postgres Vectorstore. This approach is supported and allows for better scalability and organization by user. The test test_load_index_from_storage_multiple demonstrates constructing and loading multiple indices from storage, confirming that this setup is possible ^[1][2].

To continue talking to Dosu, mention @dosu.