[Question]: vllm本地部署如何使用tools

[lullabies777]

Has this been raised before?

• [X] I have checked the GitHub README.
• [X] I have checked the Qwen documentation and cannot find an answer there.
• [X] I have searched the issues and there is not a similar one.
• [X] I confirm that this is not a bug report, a feature request, or a badcase.

Description

目前vllm部署使用tools要开enable-auto-tool-choice，但是tool-call-parser只支持hermes or mistral，简介说vllm支持tools，请问一下如何用vllm本地部署支持tools,是要自己写parser嘛

[jklj077]

Hi, please refer to our blog (search for tool calling) or documentation for tool use with Qwen2.5.

Please do check the Qwen documentation.

[lullabies777]

Hi, please refer to our blog (search for tool calling) or documentation for tool use with Qwen2.5.

Please do check the Qwen documentation.

Thanks for reply! My bad!

[lonngxiang]

使用报错：

ERROR 09-19 01:43:30 hermes_tool_parser.py:103] Error in extracting tool call from response Expecting property name enclosed in double quotes: line 2 column 2 (char 2)

调用结果显示：

Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content='\n{{"name": "add_numbers", "arguments": {"a": 1, "b": 1}}}\n', role='assistant', function_call=None, tool_calls=[]), stop_reason=None)

[lonngxiang]

Hi, please refer to our blog (search for tool calling) or documentation for tool use with Qwen2.5. Please do check the Qwen documentation.

Thanks for reply! My bad!

你这能正常运行吗

[lullabies777]

Hi, please refer to our blog (search for tool calling) or documentation for tool use with Qwen2.5. Please do check the Qwen documentation.

Thanks for reply! My bad!

你这能正常运行吗

可以运行，但是出现了和最新那个帖子一样的问题，function call是在content里面，tool_calls是空的

[lonngxiang]

Hi, please refer to our blog (search for tool calling) or documentation for tool use with Qwen2.5. Please do check the Qwen documentation.

Thanks for reply! My bad!

你这能正常运行吗

可以运行，但是出现了和最新那个帖子一样的问题，function call是在content里面，tool_calls是空的

等官方了

[jklj077]

Hi, should be fixed now, please pull the model files again (updating the tokenizer_config.json is enough). Feel free to reopen if the problem persists.

[kingxjs]

嗨，现在应该修复了，请再次拉取模型文件（更新tokenizer_config.json就足够了）。如果问题仍然存在，请随时重新打开。

May I ask if the ollama mirror has been updated, found that using 14b 7cdf5a0187d5, tool_calls is still empty

[mawenju203]

Hi, should be fixed now, please pull the model files again (updating the tokenizer_config.json is enough). Feel free to reopen if the problem persists.

你好，我这边Qwen2.5-7B-Instruct-AWQ；也出现了；代码和模型都是这周（2024年9月24号）下载的；使用是vllm=0.6.1.post2版本；

使用vllm启动的：vllm serve Qwen/Qwen2.5-7B-Instruct --enable-auto-tool-choice --tool-call-parser hermes

结果生成：