Closed Chetan8000 closed 1 year ago
You can integrate to the Langchain Agent API yourself through a Custom LLM Agent, plus some custom inference code to get the right behavior. I've just implemented it here as an example: https://github.com/paolorechia/learn-langchain/tree/main
I don't think there are currently any endpoints out there exposing to retrieve Vicuna's embeddings to integrate with other use cases though.
Thanks to the openai API, it's quite easy to integrate with langchain. A little difference is langchain could use mulit str as 'stop' param, while in fschat it's a single str. langchain would use a 2element list stop in some ReAct task, it's broken in fschat.
stop string or array Optional Defaults to null Up to 4 sequences where the API will stop generating further tokens. The returned text will not contain the stop sequence.
@oreo-yum You are absolutely correct. Which is why I had to implement a custom inference code, so I could process the stop tokens appropriately.
It was basically forked from the fschat streaming endpoint - maybe a solution is to add a new endpoint to the fschat API, that is compatible with the langchain ReAct agent framework?
We add an experimental OpenAI compatible API - https://github.com/lm-sys/FastChat#openai-compatible-restful-apis--sdk. Would this be helpful?
I tried a project lanchain-ChatGLM,its emmbedding is text2vec. Using both text2vec & LLM(chatglm), the chatbot can answer user's question according to the text materials uploaded by users. So I think maybe it can also use vicuna as LLM, and it will make greater effect
I did something similar here, but using a sentence transformer for the embeddings:
Unfortunately I have not yet added installation instructions, but I’ll do eventually, so anyone can try it.
I tried using Vicuna embeddings directly, but they didn’t perform well (at least not with my code), so I’d recommend using a different model for the embeddings part for now.
@paolorechia That's great! We could select embeddings based on https://huggingface.co/spaces/mteb/leaderboard. As the OpenAI model text-embedding-ada-002 is also listed, we could find some open source models with similar performance, by ranking by specified metric. By retrieval task might be a good choice for RetrievalQA in langchian.
An example for integrating vicuna with llama-index[0.6.6] and langchain by using OpenAI-Compatible RESTful APIs & SDK.
https://github.com/yjcyxky/chat-publications/blob/main/chatbot_vicuna.py
An example for integrating vicuna with llama-index[0.6.6] and langchain by using OpenAI-Compatible RESTful APIs & SDK.
https://github.com/yjcyxky/chat-publications/blob/main/chatbot_vicuna.py
404?
An example for integrating vicuna with llama-index[0.6.6] and langchain by using OpenAI-Compatible RESTful APIs & SDK.↳ https://github.com/yjcyxky/chat-publications/blob/main/chatbot_vicuna.py
404?
Sorry, I forgot that it's private repo. I have changed it to public.
You can use vicuna/wizardlm etc. in h2oGPT that has LangChain support: https://github.com/h2oai/h2ogpt
closed by #1246
Is there any possibility of integrating Vicuna with Lanchain. Both Vicuna and Lanchain, I believe that integrating these two powerful tools would greatly enhance their capabilities and provide users with a seamless and efficient experience