[Feature]: Is it supported Qwen2.5 tool_choice: auto?

vllm-project / vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

https://docs.vllm.ai

Apache License 2.0

30.09k stars 4.55k forks source link

[Feature]: Is it supported Qwen2.5 tool_choice: auto? #9789

Open deku0818 opened 2 weeks ago

deku0818 commented 2 weeks ago

🚀 The feature, motivation and pitch

Such as topic

Alternatives

No response

Additional context

No response

Before submitting a new issue...

[X] Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

cedonley commented 2 weeks ago

Per the qwen2.5 docs they have also trained on the Hermes style tool prompts. I set Hermes options for auto tool with qwen2.5 and it works great.

deku0818 commented 2 weeks ago

Per the qwen2.5 docs they have also trained on the Hermes style tool prompts. I set Hermes options for auto tool with qwen2.5 and it works great.根据 qwen2.5 的文档，他们还训练了 Hermes 风格的工具提示。我为 qwen2.5 设置了 Hermes 的自动工具选项，效果非常好。

How should I use it? I also used Hermes-style tool prompts, but when there is a list as a parameter during the tool invocation, an error occurs. Could you give me your startup parameters?