search

lm-sys / FastChat

An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
Apache License 2.0
36.72k stars 4.53k forks source link

How can we use vicuna for information retrievel from bunch of docs? #620

Open alan-ai-learner opened 1 year ago

alan-ai-learner commented 1 year ago

Hi @yantao0527 @infwinston @Mearman @zhisbug @jegonzal , I've got a question, I have let's say a book on NLP and I want to use Vicuna in such a way that if user asks any question, the model can answer from the book only. Is it possible using Vicuna or other llms?

Any suggestion would be very helpful! Thanks!

yantao0527 commented 1 year ago

I recommend a youtube video to learn how to use OpenAI's new GPT-4 api to 'chat' with and analyze multiple PDF files.

alan-ai-learner commented 1 year ago

@yantao0527 , i will look into it, that is my last and expensive option , but before that i was thinking is it possible with fine tuning any open source model like vicuna.

Thanks!

merrymercy commented 1 year ago

You can combine other tools (e.g. LangChain) and Vicuna's OpenAI-compatible API (https://github.com/lm-sys/FastChat#openai-compatible-restful-apis--sdk).

I think it does not work out-of-the-box now. Contributions are welcome.

Marenz commented 1 year ago

Could fine-tuning on the book be done easily? How would you need to structure the data?

hitchclimber commented 1 year ago

I would also be interested to know how one could fine-tune Vicuna with books or any other large bodies information

surak commented 1 year ago

While this is good, it's not really an issue of fastchat, right?

I would recommend you have a look at this colab on how to fine-tune llama2 as an example: https://colab.research.google.com/drive/1Ly01S--kUwkKQalE-75skalp-ftwl0fE?usp=sharing