xorbitsai / inference

Replace OpenAI GPT with another LLM in your app by changing a single line of code. Xinference gives you the freedom to use any LLM you need. With Xinference, you're empowered to run inference with any open-source language models, speech recognition models, and multimodal models, whether in the cloud, on-premises, or even on your laptop.
https://inference.readthedocs.io
Apache License 2.0
5.4k stars 438 forks source link

qwen2.5-instruct不支持tool calls #2377

Closed yebanliuying closed 1 month ago

yebanliuying commented 1 month ago

System Info / 系統信息

ubuntu22

Running Xinference with Docker? / 是否使用 Docker 运行 Xinfernece?

Version info / 版本信息

v0.15.2

The command used to start Xinference / 用以启动 xinference 的命令

正常启动

Reproduction / 复现过程

openai.BadRequestError: Error code: 400 - {'detail': "Only ['qwen1.5-chat', 'qwen1.5-moe-chat', 'qwen2-instruct', 'qwen2-moe-instruct', 'glm4-chat', 'glm4-chat-1m'] support tool calls"}

Expected behavior / 期待表现

尽快支持qwen2.5的tool calls

CEfanmin commented 1 month ago

v0.15.2

v0.15.1版本支持Qwen2.5-72B-ins的工具调用,但不支持流式返回

zmalqp189 commented 1 month ago

v0.15.2

v0.15.1版本支持Qwen2.5-72B-ins的工具调用,但不支持流式返回

image 为什么我看v0.15.1和v0.15.2的源码都是不支持工具调用的呢?qwen2.5-7b-instruct在v0.15.2还是我改了下源码才能调,但稍微复杂点的工具还是会输出token格式不正确。

Zhangtiande commented 1 month ago

15.3可以了吗,还没试过