zhanghx0905
commented
4 months ago 这个issue应该提到xinference下面，xinderence下个版本会支持，可以先试试qwen1.5 https://github.com/xorbitsai/inference/pull/1642
Closed JinCheng666 closed 3 months ago
zhanghx0905
commented
4 months ago 这个issue应该提到xinference下面，xinderence下个版本会支持，可以先试试qwen1.5 https://github.com/xorbitsai/inference/pull/1642
JinCheng666
commented
4 months ago 这个issue应该提到xinference下面，xinderence下个版本会支持，可以先试试qwen1.5 xorbitsai/inference#1642
感谢，尝试了qwen1.5-14b，确实可以正常使用。
刚升级了inference0.12.2，针对qwen2，确实不输出思考的过程了，但还是会多余地回复一些思考的中间值 @zhanghx0905
Question: 你是谁<|im_end|>
<|im_start|>assistant
Thought: I now know the final answer.
Final answer: 我是通义千问，由阿里云开发的AI助手。我被设计用来回答各种问题，提供信息和与用户进行对话。有什么我可以帮助你的吗？<|im_end|>
<|im_start|>user
Question: 获取当前时间<|im_end|>
<|im_start|>assistant
Thought: I can use m3VVjR.
Action: m3VVjR
Action Input: {}<|im_end|>
<|im_start|>function
Observation: {
"time": "2024-06-21 19:08:26 Friday"
}<|im_end|>
<|im_start|>assistant
, generate config: {'temperature': 0.0, 'tool_choice': 'auto', 'stream': True, 'stop': ['<|endoftext|>', '<|im_start|>', '<|im_end|>'], 'stop_token_ids': [151643, 151644, 151645]}
2024-06-21 19:08:33,402 xinference.core.model 38993 DEBUG After request chat, current serve request count: 0 for the model qwen:72b
2024-06-21 19:08:33,402 xinference.core.model 38993 DEBUG Leave wrapped_func, elapsed time: 0 s
INFO 06-21 19:08:33 async_llm_engine.py:564] Received request 994ce71c-2fbe-11ef-b1f7-0cda411d272a: prompt: '<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n<|im_start|>user\nAnswer the following questions as best you can. You have access to the following APIs:\n\nm3VVjR: Call this tool to interact with the m3VVjR API. What is the m3VVjR API useful for? 获取用户当前时区的时间。 Parameters: [] Format the arguments as a JSON object.\n\nUse the following format:\n\nQuestion: the input question you must answer\nThought: you should always think about what to do\nAction: the action to take, should be one of [m3VVjR]\nAction Input: the input to the action\nObservation: the result of the action\n... (this Thought/Action/Action Input/Observation can be repeated zero or more times)\nThought: I now know the final answer\nFinal Answer: the final answer to the original input question\n\nBegin!\n\nQuestion: 你是谁<|im_end|>\n<|im_start|>assistant\nThought: I now know the final answer.\nFinal answer: 我是通义千问，由阿里云开发的AI助手。我被设计用来回答各种问题，提供信息和与用户进行对话。有什么我可以帮助你的吗？<|im_end|>\n<|im_start|>user\nQuestion: 获取当前时间<|im_end|>\n<|im_start|>assistant\nThought: I can use m3VVjR.\nAction: m3VVjR\nAction Input: {}<|im_end|>\n<|im_start|>function\nObservation: {\n "time": "2024-06-21 19:08:26 Friday"\n}<|im_end|>\n<|im_start|>assistant\n', params: SamplingParams(n=1, best_of=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=0.0, top_p=1.0, top_k=-1, min_p=0.0, seed=None, use_beam_search=False, length_penalty=1.0, early_stopping=False, stop=['<|endoftext|>', '<|im_start|>', '<|im_end|>'], stop_token_ids=[151643, 151644, 151645], include_stop_str_in_output=False, ignore_eos=False, max_tokens=1024, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, truncate_prompt_tokens=None), prompt_token_ids: None, lora_request: None.
INFO 06-21 19:08:36 async_llm_engine.py:133] Finished request 994ce71c-2fbe-11ef-b1f7-0cda411d272a.
2024-06-21 19:08:36,229 xinference.model.llm.utils 38993 DEBUG Tool call content: 当前时间是2024年6月21日星期五19:08:26。, func: None, args: None
xinference升级至0.12.3后，问题解决。感谢大家
例行检查
你的版本
问题描述, 日志截图
xinference部署qwen2-72b，通过oneapi接入fastgpt，使用qwen的工具调用功能时，对话中输出了大模型的思考过程，希望不要输出大模型的思考过程。
关联issue： https://github.com/labring/FastGPT/issues/1668
版本信息：
xinference:0.12.1 fastgpt:4.8.4-fix oneapi:0.5.10 qwen2:Qwen2-72B-Instruct-GPTQ-Int4
config.json
xinference的日志：
修改配置文件，toolchoice为false，functioncall为true，无法调用工具