openai server非流式请求报错

[FuturePrayer]

模型：glm4-9b-chat q4_0量化 接口：/v1/chat/completions 请求报文：

{
    "model": "gpt-3.5-turbo",
    "stream": false,
    "messages": [
      {
        "role": "system",
        "content": "You are a helpful assistant."
      },
      {
        "role": "user",
        "content": "Hello!"
      }
    ]
  }

后台报错日志：

INFO:     183.192.87.174:0 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "C:\AI\chatglm.cpp\venv\Lib\site-packages\uvicorn\protocols\http\httptools_impl.py", line 399, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\AI\chatglm.cpp\venv\Lib\site-packages\uvicorn\middleware\proxy_headers.py", line 70, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\AI\chatglm.cpp\venv\Lib\site-packages\fastapi\applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "C:\AI\chatglm.cpp\venv\Lib\site-packages\starlette\applications.py", line 123, in __call__
    await self.middleware_stack(scope, receive, send)
  File "C:\AI\chatglm.cpp\venv\Lib\site-packages\starlette\middleware\errors.py", line 186, in __call__
    raise exc
  File "C:\AI\chatglm.cpp\venv\Lib\site-packages\starlette\middleware\errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "C:\AI\chatglm.cpp\venv\Lib\site-packages\starlette\middleware\cors.py", line 85, in __call__
    await self.app(scope, receive, send)
  File "C:\AI\chatglm.cpp\venv\Lib\site-packages\starlette\middleware\exceptions.py", line 65, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "C:\AI\chatglm.cpp\venv\Lib\site-packages\starlette\_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "C:\AI\chatglm.cpp\venv\Lib\site-packages\starlette\_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "C:\AI\chatglm.cpp\venv\Lib\site-packages\starlette\routing.py", line 756, in __call__
    await self.middleware_stack(scope, receive, send)
  File "C:\AI\chatglm.cpp\venv\Lib\site-packages\starlette\routing.py", line 776, in app
    await route.handle(scope, receive, send)
  File "C:\AI\chatglm.cpp\venv\Lib\site-packages\starlette\routing.py", line 297, in handle
    await self.app(scope, receive, send)
  File "C:\AI\chatglm.cpp\venv\Lib\site-packages\starlette\routing.py", line 77, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "C:\AI\chatglm.cpp\venv\Lib\site-packages\starlette\_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "C:\AI\chatglm.cpp\venv\Lib\site-packages\starlette\_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "C:\AI\chatglm.cpp\venv\Lib\site-packages\starlette\routing.py", line 72, in app
    response = await func(request)
               ^^^^^^^^^^^^^^^^^^^
  File "C:\AI\chatglm.cpp\venv\Lib\site-packages\fastapi\routing.py", line 278, in app
    raw_response = await run_endpoint_function(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\AI\chatglm.cpp\venv\Lib\site-packages\fastapi\routing.py", line 191, in run_endpoint_function
    return await dependant.call(**values)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\AI\chatglm.cpp\venv\Lib\site-packages\chatglm_cpp\openai_api.py", line 212, in create_chat_completion
    prompt_tokens = len(pipeline.tokenizer.encode_messages(messages, max_context_length))
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'chatglm_cpp._C.ChatGLM4Tokenizer' object has no attribute 'encode_messages'. Did you mean: 'decode_message'?

[FuturePrayer]

启动命令：

(venv) C:\Windows\System32>uvicorn chatglm_cpp.openai_api:app --host 127.0.0.1 --port 4523

[ShawJiang]

I ran it on m2 ultra and encountered the same error. The running command was "MODEL=./chatglm.cpp/models/chatglm4-ggml.bin uvicorn chatglm_cpp.openai_api:app --host 0.0.0.0 --port 8848"

[li-plus]

Fixed by #309. Will release a new version soon.

[FuturePrayer]

I ran it on m2 ultra and encountered the same error. The running command was "MODEL=./chatglm.cpp/models/chatglm4-ggml.bin uvicorn chatglm_cpp.openai_api:app --host 0.0.0.0 --port 8848"

我查看了#309的修改方式，直接修改了site-packages中的代码，把encode_messages修改为apply_chat_template，不再报错了

[ShawJiang]

I ran it on m2 ultra and encountered the same error. The running command was "MODEL=./chatglm.cpp/models/chatglm4-ggml.bin uvicorn chatglm_cpp.openai_api:app --host 0.0.0.0 --port 8848"

我查看了#309的修改方式，直接修改了site-packages中的代码，把encode_messages修改为apply_chat_template，不再报错了

我暂时也用相同的方式解决了，多谢！

[li-plus]

已经发布新版本了，可以更新下 python 包，先关了