after convert by openvino, how to support "all tool mode" ?

THUDM / GLM-4

GLM-4 series: Open Multilingual Multimodal Chat LMs | 开源多语言多模态对话模型

Apache License 2.0

4.75k stars 392 forks source link

after convert by openvino, how to support "all tool mode" ? #294

Closed AlbertG123 closed 3 months ago

AlbertG123 commented 3 months ago

Feature request / 功能建议

I just find the guide to run " python3 chat.py --model_path {your_path}/glm-4-9b-chat --max_sequence_length 4096 --device GPU" how to run “all tools mode" ?

Motivation / 动机

we need a way to run GLM4 all features on Intel CPU/GPU/NPU.

Your contribution / 您的贡献

zRzRzRzRzRzRzR commented 3 months ago

这个demo只是一个让你跑起来的办法，你需要自己将这载入模型的办法替换掉我们 openai_api_server中，自行修改代码替换掉原始vLLM的代码，然后变成OpenAI API格式测试就行