Functionalized the RAG. Side: Uses high Memory , requires optimizations - Githubissues

dvp-git / RAG_mistralai_chat_bot

A RAG chatbot application using faiss , mistral-instruct-v2.0 and all-mpnet-base

https://console.paperspace.com/dvp-git/notebook/ro0b8p0x8kny466

MIT License

1 stars 1 forks source link

Functionalized the RAG. Side: Uses high Memory , requires optimizations #5

Closed dvp-git closed 6 months ago

dvp-git commented 6 months ago

Functionalized the application. Notes:

Uses high memory due to loading of model everytime the app is run.
Using Quantized configuration 4-bit Mistral-ai-bot.

Possible enhancements :

Run a high power compute GPU in separate instance.
Use API for fetching model embeddings and generations.