会不会考虑支持多模态模型

[ehlxr]

会不会考虑支持多模态模型，比如 glm-4v-9b

[shell-nlp]

考虑支持，暂时没时间做

[shell-nlp]

会不会考虑支持多模态模型，比如 glm-4v-9b

dev版本支持了 glm-4v-9b 的 lmdeploy-pytorch 后端，欢迎体验 测试 这样安装不会报错 pip install -r requirements-dev.txt --no-deps

[shell-nlp]

后面有时间会继续完善其它 多模态模型，优先支持 中文、VLM榜单靠前、口碑好的模型

[shell-nlp]

使用样例代码： https://github.com/shell-nlp/gpt_server/blob/main/tests/test_openai_vl_chat.py

[ehlxr]

大佬，报错了 使用 pip install -r requirements-dev.txt --no-deps 安装的依赖

[shell-nlp]

今晚我看一下

---原始邮件--- 发件人: @.> 发送时间: 2024年7月22日(周一) 下午5:13 收件人: @.>; 抄送: @.**@.>; 主题: Re: [shell-nlp/gpt_server] 会不会考虑支持多模态模型 (Issue #11)

大佬，报错了image.png (view on web)

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

[shell-nlp]

大佬，报错了 使用 pip install -r requirements-dev.txt --no-deps 安装的依赖

我这里测试是正常的，我看了代码， 你这里报错是由于控制器没有注册上， 你看一下你的日志，控制器 是否启动正常

[ehlxr]

大佬，报错了 使用 pip install -r requirements-dev.txt --no-deps 安装的依赖

我这里测试是正常的，我看了代码， 你这里报错是由于控制器没有注册上， 你看一下你的日志，控制器 是否启动正常

这是完整的日志

(.venv) (base) root@ubuntu:/home/ubuntu/lxr/gpt_server-main# sh gpt_server/script/start.sh
/home/ubuntu/lxr/gpt_server-main/gpt_server
2024-07-23 01:14:38.558 | DEBUG    | gpt_server.utils:delete_log:70 - logs_path: /home/ubuntu/lxr/gpt_server-main/logs

2024-07-23 01:14:38.575 | INFO     | gpt_server.utils:run_cmd:9 - 执行命令如下：
python -m gpt_server.serving.start_api_server --host 0.0.0.0 --port 8082

2024-07-23 01:14:38.604 | INFO     | gpt_server.utils:run_cmd:9 - 执行命令如下：
CUDA_VISIBLE_DEVICES=0,1 python -m gpt_server.model_worker.chatglm --num_gpus 2 --model_name_or_path /home/ubuntu/lxr/models/glm-4v-9b/ --model_names glm-4v --backend lmdeploy-pytorch

2024-07-23 01:14:38.665 | INFO     | gpt_server.utils:run_cmd:9 - 执行命令如下：
python -m fastchat.serve.controller

2024-07-23 01:14:38.666 | INFO     | gpt_server.utils:run_cmd:9 - 执行命令如下：
python -m gpt_server.serving.openai_api_server --host 0.0.0.0 --port 8082

2024-07-23 01:14:39 | INFO | controller | args: Namespace(host='localhost', port=21001, dispatch_method='shortest_queue', ssl=False)
2024-07-23 01:14:39 | ERROR | stderr | INFO:     Started server process [3326873]
2024-07-23 01:14:39 | ERROR | stderr | INFO:     Waiting for application startup.
2024-07-23 01:14:39 | ERROR | stderr | INFO:     Application startup complete.
2024-07-23 01:14:39 | ERROR | stderr | INFO:     Uvicorn running on http://localhost:21001 (Press CTRL+C to quit)
2024-07-23 01:14:39 | INFO | openai_api_server | args: Namespace(host='0.0.0.0', port=8082, controller_address='http://localhost:21001', allow_credentials=False, allowed_origins=['*'], allowed_methods=['*'], allowed_headers=['*'], api_keys=None, ssl=False)
2024-07-23 01:14:39 | ERROR | stderr | INFO:     Started server process [3326875]
2024-07-23 01:14:39 | ERROR | stderr | INFO:     Waiting for application startup.
2024-07-23 01:14:39 | ERROR | stderr | INFO:     Application startup complete.
2024-07-23 01:14:39 | ERROR | stderr | INFO:     Uvicorn running on http://0.0.0.0:8082 (Press CTRL+C to quit)
INFO:     Started server process [3326869]
INFO:     Waiting for application startup.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
2024-07-23 01:14:42.220 | INFO     | gpt_server.model_worker.base:load_model_tokenizer:109 - ChatGLMWorker 使用 LMDeploy 后端
2024-07-23 01:14:42.220 | INFO     | gpt_server.model_backend.lmdeploy_backend:__init__:22 - 后端 pytorch
2024-07-23 01:14:42.236 | INFO     | gpt_server.model_backend.lmdeploy_backend:__init__:32 - 模型架构：vlm
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
2024-07-23 01:15:00.407 | INFO     | gpt_server.model_worker.base:load_model_tokenizer:127 - load_model_tokenizer 完成
2024-07-23 01:15:00.408 | INFO     | gpt_server.model_worker.base:get_context_length:73 - 模型配置：
2024-07-23 01:15:00.409 | INFO     | gpt_server.model_worker.base:__init__:57 - Loading the model ['glm-4v'] on worker 0001e8d3 ...
ERROR:    Traceback (most recent call last):
  File "/home/ubuntu/lxr/gpt_server-main/.venv/lib/python3.11/site-packages/starlette/routing.py", line 732, in lifespan
    async with self.lifespan_context(app) as maybe_state:
  File "/home/ubuntu/lxr/gpt_server-main/.venv/lib/python3.11/site-packages/starlette/routing.py", line 608, in __aenter__
    await self._router.startup()
  File "/home/ubuntu/lxr/gpt_server-main/.venv/lib/python3.11/site-packages/starlette/routing.py", line 709, in startup
    await handler()
  File "/home/ubuntu/lxr/gpt_server-main/gpt_server/model_worker/base.py", line 199, in startup
    worker = cls.get_worker(
             ^^^^^^^^^^^^^^^
  File "/home/ubuntu/lxr/gpt_server-main/gpt_server/model_worker/base.py", line 153, in get_worker
    worker = cls(
             ^^^^
  File "/home/ubuntu/lxr/gpt_server-main/gpt_server/model_worker/chatglm.py", line 23, in __init__
    super().__init__(
  File "/home/ubuntu/lxr/gpt_server-main/gpt_server/model_worker/base.py", line 58, in __init__
    self.init_heart_beat()
  File "/home/ubuntu/lxr/gpt_server-main/.venv/lib/python3.11/site-packages/fastchat/serve/base_model_worker.py", line 81, in init_heart_beat
    self.register_to_controller()
  File "/home/ubuntu/lxr/gpt_server-main/.venv/lib/python3.11/site-packages/fastchat/serve/base_model_worker.py", line 100, in register_to_controller
    assert r.status_code == 200
           ^^^^^^^^^^^^^^^^^^^^
AssertionError

ERROR:    Application startup failed. Exiting.

配置文件：

以下是 huggingface 下载模型的截图：

[shell-nlp]

看日志是没有问题的，你可以加我 微信

[shell-nlp]

如果问题已解决请关闭 issue