Openai style api for open large language models, using LLMs just as chatgpt! Support for LLaMA, LLaMA-2, BLOOM, Falcon, Baichuan, Qwen, Xverse, SqlCoder, CodeLLaMA, ChatGLM, ChatGLM2, ChatGLM3 etc. 开源大模型的统一后端接口
提交前必须检查以下项目 | The following items must be checked before submission
[X] 请确保使用的是仓库最新代码（git pull），一些问题已被解决和修复。 | Make sure you are using the latest code from the repository (git pull), some issues have already been addressed and fixed.
[X] 我已阅读项目文档和FAQ章节并且已在Issue中对问题进行了搜索，没有找到相似问题和解决方案 | I have searched the existing issues / discussions
lzhfe
commented
7 months ago
提交前必须检查以下项目 | The following items must be checked before submission
问题类型 | Type of problem
None
操作系统 | Operating system
None
详细描述问题 | Detailed description of the problem
engine：vllm 参数：
{"messages": [{"content": "计算6*9+3.5-9.7", "role": "user"}], "model": "xxx-32k", "frequency_penalty": 0.0, "functions": [{"name": "eval_expression", "description": "A tool for evaluating an math expression.", "parameters": {"type": "object", "properties": {"expression": {"type": "string", "description": ""}}, "required": ["expression"]}}], "max_tokens": 512, "n": 1, "presence_penalty": 0.0, "temperature": 0.0, "top_p": 1.0, "user": "xx", "stream": true}返回：
返回识别到要调用function，返回结果如上图所示，有2个问题： 1、出现2个finish_reason 2、React识别到需要调用function了，但finish_reason仍为Stop，期望是 function_call
Dependencies
No response
运行日志或截图 | Runtime logs or screenshots
No response