[help] Vicuna with local document

[qkyyds666]

Hi, I have a local document and I want to know how to use langchain and vicuna to read my local document and then I can talk to vicuna to discuss about the content of the document？

[paolorechia]

Hi,

I think you need an embedding function to use Vicuna to read the document as embeddings, and then store into a Vector Store.

Then you should be able load it as memory for your chain / agent.

I saw a discussion in the fastchat repository about adding an API endpoint for embedding - I haven’t keep track of how it went - if you’re able to implement it, check the issues there.

I’ll try looking into this if I get some time, but I’m not 100% how to extract the embeddings, so will have to do some reading on my side

qkyyds666 @.***> schrieb am Sa. 22. Apr. 2023 um 16:39:

Hi, I have a local document and I want to know how to use langchain and vicuna to read my local document and then I can talk to vicuna to discuss about the content of the document？

— Reply to this email directly, view it on GitHub https://github.com/paolorechia/learn-langchain/issues/1, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJDFZ34TFEEY6J5TVX6HH3XCPURXANCNFSM6AAAAAAXH4C7R4 . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[paolorechia]

Here's the issue that I mentioned, just found it again:

https://github.com/lm-sys/FastChat/issues/235

There are people working on it, so we might have this embedding API available somewhat soon.

Another option you have is fine tuning your Vicuna model into a LoRA using your document, but I think the vector store might work better.

[paolorechia]

Hi, @qkyyds666, I've updated the main branch to include an experimental /embedding endpoint, which I have no idea whether it's working as intended :goberserk:

If I get some additional time, I'll look into setting up a vector store locally and testing this (https://python.langchain.com/en/latest/modules/memory/types/vectorstore_retriever_memory.html). Can't promise when though as I'm a bit low on time.

[qkyyds666]

Amazing👀

[paolorechia]

It seems like the next steps would include:

1. Load and split the input text into chunks
2. Use the embedding function to transform chunks into a vector
3. Save it to a Vector Store
4. When answering a question, query the Vector Store and send the best results back to the prompt

Source:

https://python.langchain.com/en/latest/modules/chains/index_examples/qa_with_sources.html

[paolorechia]

Hey, I've merged an example using Chroma to read a local .txt file and query it:

https://github.com/paolorechia/learn-langchain/pull/3

I had to modify the embedding function to use the mean, otherwise it was pure nonsense. The performance is not that great though, maybe the book I've used (Law of Plato) is too difficult for the similarity search.

I'm wondering if using another model of similarity search wouldn't perform better.

I'm closing this, if you still need help let me know.