search

adnanbhanji / EU-Chatbot-Reco-System

MIT License
0 stars 1 forks source link

LLM Chosen: Mistral. Connect to Mistral 7B locally first. #43

Closed adnanbhanji closed 6 months ago

adnanbhanji commented 6 months ago
Screenshot 2024-03-29 at 2 10 16 in the afternoon

The size of this model is around 4 GB, and due to size constraints, we found a workaround on downloading the model locally using code, which was much more efficient in loading time and integrating with our RAG implementation.

adnanbhanji commented 6 months ago

Changed the model as this quantized model didn't give the best results, went for a bigger model which took longer to respond but gave more accurate answers.