Could not reach the server

AIAnytime / Llama2-Medical-Chatbot

This is a medical bot built using Llama2 and Sentence Transformers. The bot is powered by Langchain and Chainlit. The bot runs on a decent CPU machine with a minimum of 16GB of RAM.

MIT License

296 stars 218 forks source link

Could not reach the server #12

Open Ezyweb-uk opened 12 months ago

Ezyweb-uk commented 12 months ago

Nice project and video!

As an LLM newbie I might be being too optimistic trying to run this with the Llama 2 quantized model _llama-2-7b-chat.Q4_KM.gguf on a cpu with only 8GB RAM nominal. The Chainlit page loads and after entering a question after a while it appears to timeout with the message "Could not reach the server". Should I increase the session_timeout parameter in config.toml or run some of the code in async mode?

siddhant250502 commented 7 months ago

same issue any solutions @Ezyweb-uk ?