[Bug]: TypeError: 'ChatMessage' object is not subscriptable when MODEL_PATH=openbmb/MiniCPM3-4B

cxgreat2014 commented 2 months ago

Is there an existing issue ? / 是否已有相关的 issue ?

[X] I have searched, and there is no existing issue. / 我已经搜索过了，没有相关的 issue。

Describe the bug / 描述这个 bug

无法使用openai_api_server_demo运行MiniCPM3-4B模型

To Reproduce / 如何复现

设置环境变量MODEL_PATH=openbmb/MiniCPM3-4B
运行openai_api_server_demo.py
运行openai_api_request_demo.py
报错TypeError: 'ChatMessage' object is not subscriptable

Expected behavior / 期望的结果

能够正常对话

Screenshots / 截图

minicpm-1  | 
minicpm-1  | ==========
minicpm-1  | == CUDA ==
minicpm-1  | ==========
minicpm-1  |
minicpm-1  | CUDA Version 12.1.1
minicpm-1  |
minicpm-1  | Container image Copyright (c) 2016-2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
minicpm-1  |
minicpm-1  | This container image and its contents are governed by the NVIDIA Deep Learning Container License.
minicpm-1  | By pulling and using the container, you accept the terms and conditions of this license:
minicpm-1  | https://developer.nvidia.com/ngc/nvidia-deep-learning-container-license
minicpm-1  |
minicpm-1  | A copy of this license is made available in this container at /NGC-DL-CONTAINER-LICENSE for your convenience.
minicpm-1  |
minicpm-1  | A new version of the following files was downloaded from https://huggingface.co/openbmb/MiniCPM3-4B:
minicpm-1  | - configuration_minicpm.py
minicpm-1  | . Make sure to double-check they do not contain any added malicious code. To avoid downloading new versions of the code file, you can pin a revision.
minicpm-1  | A new version of the following files was downloaded from https://huggingface.co/openbmb/MiniCPM3-4B:
minicpm-1  | - modeling_minicpm.py
minicpm-1  | . Make sure to double-check they do not contain any added malicious code. To avoid downloading new versions of the code file, you can pin a revision.
minicpm-1  | INFO:     Started server process [27]
minicpm-1  | INFO:     Waiting for application startup.
minicpm-1  | INFO:     Application startup complete.
minicpm-1  | INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
minicpm-1  | 2024-09-07 03:50:30.021 | DEBUG    | __main__:create_chat_completion:240 - ==== request ====
minicpm-1  | {'messages': [ChatMessage(role='system', content='\nCurrent model: gpt-3.5-turbo\nCurrent date: 2024-09-07T03:50:30.016Z\n\nYou are a helpful assistant. You can help me by answering my que
stions. You can also ask me questions.', name=None), ChatMessage(role='user', content='小明的爷爷有三个孙子，大孙子叫大毛，三孙子叫三毛，二孙子叫什么？', name=None)], 'temperature': 0.7, 'top_p': 1.0, 'max_tokens': 2048, 'echo': False, 'repetition_penalty': 1.1, 'tools': None}
minicpm-1  | INFO:     172.21.0.1:35712 - "POST /v1/chat/completions HTTP/1.1" 200 OK
minicpm-1  | ERROR:    Exception in ASGI application
minicpm-1  | Traceback (most recent call last):
minicpm-1  |   File "/opt/conda/lib/python3.12/site-packages/sse_starlette/sse.py", line 289, in __call__
minicpm-1  |     await wrap(partial(self.listen_for_disconnect, receive))
minicpm-1  |   File "/opt/conda/lib/python3.12/site-packages/sse_starlette/sse.py", line 278, in wrap
minicpm-1  |     await func()
minicpm-1  |   File "/opt/conda/lib/python3.12/site-packages/sse_starlette/sse.py", line 228, in listen_for_disconnect
minicpm-1  |     message = await receive()
minicpm-1  |               ^^^^^^^^^^^^^^^
minicpm-1  |   File "/opt/conda/lib/python3.12/site-packages/uvicorn/protocols/http/h11_impl.py", line 534, in receive
minicpm-1  |     await self.message_event.wait()
minicpm-1  |   File "/opt/conda/lib/python3.12/asyncio/locks.py", line 212, in wait
minicpm-1  |     await fut
minicpm-1  | asyncio.exceptions.CancelledError: Cancelled by cancel scope 7f4a63169970
minicpm-1  |
minicpm-1  | During handling of the above exception, another exception occurred:
minicpm-1  |
minicpm-1  |   + Exception Group Traceback (most recent call last):
minicpm-1  |   |   File "/opt/conda/lib/python3.12/site-packages/uvicorn/protocols/http/h11_impl.py", line 406, in run_asgi
minicpm-1  |   |     result = await app(  # type: ignore[func-returns-value]
minicpm-1  |   |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
minicpm-1  |   |   File "/opt/conda/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", line 70, in __call__
minicpm-1  |   |     return await self.app(scope, receive, send)
minicpm-1  |   |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
minicpm-1  |   |   File "/opt/conda/lib/python3.12/site-packages/fastapi/applications.py", line 1054, in __call__
minicpm-1  |   |     await super().__call__(scope, receive, send)
minicpm-1  |   |   File "/opt/conda/lib/python3.12/site-packages/starlette/applications.py", line 113, in __call__
minicpm-1  |   |     await self.middleware_stack(scope, receive, send)
minicpm-1  |   |   File "/opt/conda/lib/python3.12/site-packages/starlette/middleware/errors.py", line 187, in __call__
minicpm-1  |   |     raise exc
minicpm-1  |   |   File "/opt/conda/lib/python3.12/site-packages/starlette/middleware/errors.py", line 165, in __call__
minicpm-1  |   |     await self.app(scope, receive, _send)
minicpm-1  |   |   File "/opt/conda/lib/python3.12/site-packages/starlette/middleware/cors.py", line 85, in __call__
minicpm-1  |   |     await self.app(scope, receive, send)
minicpm-1  |   |   File "/opt/conda/lib/python3.12/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
minicpm-1  |   |     await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
minicpm-1  |   |   File "/opt/conda/lib/python3.12/site-packages/starlette/_exception_handler.py", line 62, in wrapped_app
minicpm-1  |   |     raise exc
minicpm-1  |   |   File "/opt/conda/lib/python3.12/site-packages/starlette/_exception_handler.py", line 51, in wrapped_app
minicpm-1  |   |     await app(scope, receive, sender)
minicpm-1  |   |   File "/opt/conda/lib/python3.12/site-packages/starlette/routing.py", line 715, in __call__
minicpm-1  |   |     await self.middleware_stack(scope, receive, send)
minicpm-1  |   |   File "/opt/conda/lib/python3.12/site-packages/starlette/routing.py", line 735, in app
minicpm-1  |   |     await route.handle(scope, receive, send)
minicpm-1  |   |   File "/opt/conda/lib/python3.12/site-packages/starlette/routing.py", line 288, in handle
minicpm-1  |   |     await self.app(scope, receive, send)
minicpm-1  |   |   File "/opt/conda/lib/python3.12/site-packages/starlette/routing.py", line 76, in app
minicpm-1  |   |     await wrap_app_handling_exceptions(app, request)(scope, receive, send)
minicpm-1  |   |   File "/opt/conda/lib/python3.12/site-packages/starlette/_exception_handler.py", line 62, in wrapped_app
minicpm-1  |   |     raise exc
minicpm-1  |   |   File "/opt/conda/lib/python3.12/site-packages/starlette/_exception_handler.py", line 51, in wrapped_app
minicpm-1  |   |     await app(scope, receive, sender)
minicpm-1  |   |   File "/opt/conda/lib/python3.12/site-packages/starlette/routing.py", line 74, in app
minicpm-1  |   |     await response(scope, receive, send)
minicpm-1  |   |   File "/opt/conda/lib/python3.12/site-packages/sse_starlette/sse.py", line 275, in __call__
minicpm-1  |   |     async with anyio.create_task_group() as task_group:
minicpm-1  |   |   File "/opt/conda/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 680, in __aexit__
minicpm-1  |   |     raise BaseExceptionGroup(
minicpm-1  |   | ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
minicpm-1  |   +-+---------------- 1 ----------------
minicpm-1  |     | Traceback (most recent call last):
minicpm-1  |     |   File "/opt/conda/lib/python3.12/site-packages/sse_starlette/sse.py", line 278, in wrap
minicpm-1  |     |     await func()
minicpm-1  |     |   File "/opt/conda/lib/python3.12/site-packages/sse_starlette/sse.py", line 258, in stream_response
minicpm-1  |     |     async for data in self.body_iterator:
minicpm-1  |     |   File "/app/openai_api_server_demo.py", line 245, in stream_response
minicpm-1  |     |     for new_response in generate_minicpm(model, tokenizer, gen_params):
minicpm-1  |     |   File "/app/openai_api_server_demo.py", line 153, in generate_minicpm
minicpm-1  |     |     inputs = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=False)
minicpm-1  |     |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
minicpm-1  |     |   File "/cache/huggingface/modules/transformers_modules/openbmb/MiniCPM3-4B/75f9f1097d9d66d11f37fff49210bf940455f8ac/tokenization_minicpm.py", line 45, in apply_chat_template        
minicpm-1  |     |     check_messages(conversation, tools)
minicpm-1  |     |   File "/cache/huggingface/modules/transformers_modules/openbmb/MiniCPM3-4B/75f9f1097d9d66d11f37fff49210bf940455f8ac/tokenization_minicpm.py", line 183, in check_messages
minicpm-1  |     |     if message["role"] == "assistant" and "tool_calls" in message and len(message["tool_calls"]) > 0:
minicpm-1  |     |        ~~~~~~~^^^^^^^^
minicpm-1  |     | TypeError: 'ChatMessage' object is not subscriptable
minicpm-1  |     +------------------------------------

Environment / 环境

- OS: [Docker Ubuntu 22.04]
- Pytorch: [torch '2.4.1+cu121']
- CUDA: [CUDA 12.1.1]
- Device: [RTX3090]

Additional context / 其他信息

No response

exthirteen commented 2 months ago

你这个问题我好像遇到过，使用9/5最新发布的MiniCPM3-4B模型，不能用该项目demo/openai_api_demo/openai_api_server_demo.py启动，可以看到这个提交日期是5个月以前，看样子是不兼容最新的模型文件。你需要拉取最新代码，使用该项目目录：demo/function_call/openai_api_server.py 这个文件来启动openapi服务。不过好像开发者还是没更新全，依旧启动还是有些问题，可能与conda相关依赖有关

zh-zheng commented 2 months ago

你这个问题我好像遇到过，使用9/5最新发布的MiniCPM3-4B模型，不能用该项目demo/openai_api_demo/openai_api_server_demo.py启动，可以看到这个提交日期是5个月以前，看样子是不兼容最新的模型文件。你需要拉取最新代码，使用该项目目录：demo/function_call/openai_api_server.py 这个文件来启动openapi服务。不过好像开发者还是没更新全，依旧启动还是有些问题，可能与conda相关依赖有关

对的，MiniCPM3-4B 请使用demo/function_call/openai_api_server.py启动openai服务。如果启动有问题，请附上报错信息。此外，由于vLLM已经默认支持MiniCPM-2B，如果要启动openai服务，可以直接使用vLLM。

OpenBMB / MiniCPM