Enhancements required to achieve better results - Githubissues

dvp-git / RAG_mistralai_chat_bot

A RAG chatbot application using faiss , mistral-instruct-v2.0 and all-mpnet-base

https://console.paperspace.com/dvp-git/notebook/ro0b8p0x8kny466

MIT License

1 stars 1 forks source link

Enhancements required to achieve better results #6

Open dvp-git opened 6 months ago

dvp-git commented 6 months ago

Running the app takes some time to load the model into memory , and since we're using quantized version, llm.to('cuda') is not made use of.

The answers from the RAG are pretty decent given that the prompt is structured well. Below is a screenshot of the 2 shot learning 👍🏽

If the context is not provided, it does reply stating that it does not have information to answer the question.

I've not evaluated using any benchmarks yet.

Evaluation of GPU requirements to be checked since the task is slow during loading of embedding and and docs embedding generation. However the one of the pdf's is a textbook so I wonder if that is bottleneck.

Future tasks:

A chat app using gradio.
Enhancements to RAG maybe embedding only keyphrase using Keybert and checking if response is good or bad ( experimental)

dvp-git commented 6 months ago

Memory usage is quite high , believe this is because of document embeddings

Sample: