Can it support qwen or glm-3?

[seoml]

The current list of supported models is almost unable to conduct Chinese conversations.

GPU：A770

models：microsoft---Phi-3-mini-4k-instruct

[qiacheng]

hi @seoml,

yes, it supports Qwen, Qwen2 and chatglm. however for model change in the current version we need to clone the model from huggingface and drop in the models/llm/checkpoints folder and you should be able to get Qwen2-7B-Insturct up and running.

In the future versions we will be exploring options to add gguf support for a better user experience

[brownplayer]

It would be interesting to add files in gguf format