Can I hook it up to a retrieval system for a document chatbot?

[adarshxs]

I want to use the instructor-xl text embedding model and use FAISS to create and retrieve from a vector store. Sort of a chatbot for documents or a domain specific chatbot. Any ideas on how I can do it?

[bulletproofmedic]

I would like to do the same, I'll try to mess around with it over the next week or two and see if I can't figure something out. Hopefully someone more knowledgeable can beat me to it though.

[adarshxs]

@bulletproofmedic do let me know if you get somewhere lol thanks

[akareen]

Working on the same will update if I can make any progress

[enesdikmen]

I am working on the same thing. Currently when I make the similarity search FAISS will always return the same vectors. I am using the same setup I have used for other embedding models. I guess this has something to do with the input format of encode() function. We give a list ["instruction", "text"]. Similarly when we do search on FAISS db it returns 2 arrays like this as distances:"[[0.47456202 0.49658185 0.5015992 ] [0.398577 0.4358151 0.4369905 ]]" (For k=3).

[loganlebanoff]

Any progress on this? I'm also interested in hooking up retrieval to the UI

[enesdikmen]

Hi there, a was able to get this working. Actually I ended up using another embedding model (BAAI/bge-large-en) but instructor-xl was working fine too.

Initalize the model:

model =  INSTRUCTOR('hkunlp/instructor-xl')

To get the embeddings of a query:

embedding = model.encode([["Represent the question for retrieving supporting documents: ", "Your query here"]], normalize_embeddings=True)

Then making similarity search on FAISS index:

//first load your index
index = faiss.read_index('path/to/your/index_file')

//make the search
D, I = index.search(embedding.astype('float32'), 3)

[gururise]

Hi there, a was able to get this working. Actually I ended up using another embedding model (BAAI/bge-large-en) but instructor-xl was working fine too.

Initalize the model:

model =  INSTRUCTOR('hkunlp/instructor-xl')

To get the embeddings of a query:

embedding = model.encode([["Represent the question for retrieving supporting documents: ", "Your query here"]], normalize_embeddings=True)

Then making similarity search on FAISS index:

//first load your index
index = faiss.read_index('path/to/your/index_file')

//make the search
D, I = index.search(embedding.astype('float32'), 3)

Where did you make these changes?

[j-dominguez9]

I'm also looking to implement this. Is there a place where a python script can take an input, be processed by Retrieval system, then send over output to model, as usual? I'm thinking maybe setting up a REST API for the python script and have the src/lib/server/generateFromDefaultEndpoint.ts file call that endpoint, so that the "message" includes the retrieved chunks as well.

[rajasblack]

Has anybody designed a chat agent and retrieval system from vector store db's like FAISS, ChromaDB?