search

edgar971 / open-chat

A self-hosted, offline, ChatGPT-like chatbot with different LLM support. 100% private, with no data leaving your device.
MIT License
64 stars 8 forks source link

Multiple issues #8

Open JAKAMI99 opened 10 months ago

JAKAMI99 commented 10 months ago

Hey! The default model works so far. I would love, to be able, to use other language models in the GGML format. Sadly, the container uses a VARIABLE to set a specific file. Instead of this, it would be awesome if you could just place other models in the "/models" folder, and change them in the webgui. I tried exactly this, but sadly this didn't work. I only could get it working when changing the "MODEL" variable, but then the model selector in the webgui would be blank, and I would be forced to use this one all the time. Maybe there is an easy way to implement the use of multiple models? :)

Thanks tho!

RolandVessalius commented 10 months ago

Same issues here. I've tried different GGML model and no luck :/

OllumHD commented 10 months ago

Same here, I can only change the model by changing the MODEL variable sadly. Atleast it kinda works after that though

AnterosOberon commented 10 months ago

hey at least it finally loads.. I too would love to be able to load additional models and additional datasets. All in good time. Thankful its working now

edgar971 commented 10 months ago

Thanks for the feedback. Let me look into this request. The main thing would be to load the selected model in VRAM to avoid running out of memory.