Can not use tool and function-call?

[chenkaiC4]

同样关注这个能力，是否有这部分能力的强化？对于企业应用，这块能力太重要了

[vicguo0724]

附议

[qizhanchan]

Need function calling too.

[ftgreat]

同样关注这个能力，是否有这部分能力的强化？对于企业应用，这块能力太重要了

请问国内哪些模型这个能力比较强，谢谢

[luofuli]

Up to your guys' demands. Please vote here and tell us why you want API support for function calling.

[qizhanchan]

To build agents with the ability of LLMs, such as Exec SQLs, Fetch data from API.

[qizhanchan]

I found something interesting that DeepSeek has magic power to match function. Here's my prompt from langchain's idea

You have access to the following tools:
[
    {
        "name": "get_location",
        "description": "get detail info of a specific location",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "The city and state, e.g. San Francisco, CA"
                }
            }
        },
        "required": [
            "location"
        ]
    },
    {
        "name": "get_math_expression_result",
        "description": "get result info of a given math expression",
        "parameters": {
            "type": "object",
            "properties": {
                "expression": {
                    "type": "string",
                    "description": "The math expression, e.g. 1+2*3, sin(a)"
                }
            }
        },
        "required": [
            "expression"
        ]
    }
]

Give priority to use a tool for zero or more times, respond with a JSON object with the following structure: 
{
    "tool": <name of the called tool given above>,
    "tool_input": <parameters for the tool matching the above JSON schema>
}
If no matched tool, just respond directly

then you can ask it

梧桐山在哪里？广州塔在哪里？7863*41是多少？golang 基础类型都有哪些？

Model will match 3 function calls and output the last answer directly coz no matched tool.

[qizhanchan]

But the emulated system prompt takes no effects in the 2nd round user message contents, my solution is move the tool related content and append it to the user content.

[vicguo0724]

fastgpt构建agent

[NaMoCv]

json_object应该比较好做，我觉得应该先支持。用户能少写点代码。

[yhyu13]

Not at all, need custom fine tuning.

Not even the deepseek-chat api has function calling abilities.

[dangnhdev]

Up to your guys' demands. Please vote here and tell us why you want API support for function calling.

• Get and introspect SQL schema on-demand in code development. It changes a lot.
• Some tools like TypingMind have their own chat agent, which can render result dynamically (HTML JS, Mermaid chart, search etc...), they use LLM's function calling to trigger that.

[dangnhdev]

* Get and introspect SQL schema on-demand in code development. It changes a lot.

I've made a ChatGPT action which basically use this function calling feature to auto get latest code files on development and generate necessary changes based on my question. So no more copy paste.

[Zhenyi-Wang]

工具调用可以扩展ai能力，比如网页搜索、页面读取、代码执行等，特别是现在超长上下文的情况下，可以极大扩展应用场景。

我试过使用类似FastGPT之类的框架（以prompt形式模拟function call），但代价是会对提示词和聊天历史造成混乱，算是堪堪够用。

虽然其他原生支持tools或者funtion的模型会好很多，但是性价比就……

[luochen1990]

不是说已经支持了么？ 这个issue 是不是可以关了？

[luochen1990]

The current cloud-based DeepSeek already supports function calls, and I have tested that it is indeed functional. However, it seems that the version supporting function calls has not yet been open-sourced. I wonder if there are any plans for this?

[caomingpei]

Until my post, I believe the deepseek didn't support the funciton call. I am working on extending the compatibility of an open-ai application. BUT when I use the deepseek api, I still get the following:

No function_call in message 
[{\"text\":\"Action: clarification\\nPayload: {\\\"message\\\": \\\"Hello! How can I assist you today with Zotero?\\\"}\",\"message\":{\"lc\":1,\"type\":\"constructor\",\"id\":[\"langchain_core\",\"messages\",\"AIMessage\"],\"kwargs\":{\"content\":\"Action: clarification\\nPayload: {\\\"message\\\": \\\"Hello! How can I assist you today with Zotero?\\\"}\",\"additional_kwargs\":{},\"response_metadata\":{\"tokenUsage\":{\"completionTokens\":24,\"promptTokens\":221,\"totalTokens\":245},\"finish_reason\":\"stop\"}}},\"generationInfo\":{\"finish_reason\":\"stop\"}}]

[luochen1990]

I'm using OpenAI API via Python SDK, and the function call works well.

[caomingpei]

I'm using OpenAI API via Python SDK, and the function call works well.

Yeah, you are right. There must be other reasons lead to my error. https://platform.deepseek.com/api-docs/function_calling/

[gkm0120]

The official api supports function call, but does the open source DeepSeek-V2-Chat-0628 support function call, or when is it planned to support function call?

[EvoNexusX]

官方api本身可以进行function calling， 但是似乎自己本地的模型在function calling时总是匹配不到

[sammcj]

If you're using Ollama give this Modelfile a go, I've been using it for tool use with DSCv2lite:

# deepseek-coder-v2-lite-toolcalling:q8_0

FROM deepseek-coder-v2-lite-instruct:q8_0

TEMPLATE """{{ if .System }}System: {{ .System }}

{{ end }}{{ if .Tools }}Available tools:
{{ range .Tools }}
{{ . }}
{{ end }}

When using a tool, you MUST respond with a JSON object in this format without any additional text:
{"name": "tool_name", "arguments": {"arg1": "value1", "arg2": "value2"}}

{{ end }}{{ range .Messages }}{{ if eq .Role "user" }}User: {{ .Content }}

{{ else if eq .Role "assistant" }}Assistant: {{ if .ToolCalls }}{{ range .ToolCalls }}{"name": "{{ .Function.Name }}", "arguments": {{ .Function.Arguments }}}
{{ end }}{{ else }}{{ .Content }}{{ end }}<｜end▁of▁sentence｜>

{{ else if eq .Role "tool" }}Tool output: {{ .Content }}

{{ end }}{{ end }}{{ if not .Messages }}{{ if .Prompt }}User: {{ .Prompt }}

{{ end }}Assistant: {{ .Response }}<｜end▁of▁sentence｜>{{ end }}"""

SYSTEM """
You are an advanced AI coding assistant, specifically designed to help with complex programming tasks, tool use, code analysis, and software architecture design. Your primary focus is on providing expert-level assistance in coding, with a special emphasis on using tool-calling capabilities when necessary. Here are your key characteristics and instructions:

1. Coding Expertise:
  - You have deep knowledge of multiple programming languages, software design patterns, and best practices.
  - Provide detailed, accurate, and efficient code solutions without additional explanations or conversational dialogue unless requested by the user.
  - When suggesting code changes, consider scalability, maintainability, and performance implications.

2. Tool Usage:
  - You have access to various tools that can assist in completing tasks. Always consider if a tool can help in your current task.
  - When you decide to use a tool, you must format your response as a JSON object:
    {"name": "tool_name", "arguments": {"arg1": "value1", "arg2": "value2"}}
  - Common tools include but are not limited to:
    - `view_file`: To examine the contents of a specific file
    - `modify_code`: To suggest changes to existing code
    - `create_file`: To create new files with specified content
    - `ask_followup_question`: To request more information from the user
    - `attempt_completion`: To indicate that you've completed the assigned task

3. Task Approach:
  - Break down complex tasks into smaller, manageable steps unless requested to solve the task at once.
  - If a task is large or complex, outline your approach before diving into details unless using a tool.
  - Use tools to gather necessary information before proposing solutions.

4. Code Analysis and Refactoring:
  - When analysing existing code, consider its structure, efficiency, and adherence to best practices.
  - Suggest refactoring when you see opportunities for improvement, explaining the benefits of your suggestions unless using a tool.
  - If you encounter or anticipate potential errors, explain them clearly and suggest solutions unless using a tool.
  - When providing code solutions, include relevant comments to explain complex logic.
  - Adhere to coding standards and best practices specific to each programming language or framework.
  - Suggest optimisations and improvements where applicable.

5. Clarity and Communication:
  - Explain your reasoning and decisions clearly, especially when suggesting architectural changes or complex solutions unless using a tool.
  - If you're unsure about any aspect of the task or need more information, use the `ask_followup_question` tool to clarify.

Remember, your primary goal is to assist with coding tasks and tool use efficiently and effectively. Utilise your tool-calling capabilities wisely to enhance your problem-solving and code generation abilities.
"""

PARAMETER stop "User:"
PARAMETER stop "Assistant:"
PARAMETER stop "<｜end▁of▁sentence｜>"

PARAMETER num_ctx 16384
PARAMETER temperature 0.1

[codefromthecrypt]

@sammcj do you mind updating this for deepseek-v2? I switched the FROM and this ended up returning quoted json instead of what models like mistral

response from v1/chat/completions using this modelfile using deepseek-v2 (doesn't work):

{
  "id": "chatcmpl-398",
  "object": "chat.completion",
  "created": 1725784785,
  "model": "deepseek",
  "system_fingerprint": "fp_ollama",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "{\n  \"type\": \"function\",\n  \"function\": {\n    \"name\": \"get_password\",\n    \"description\": \"Return the password for authentication\"\n  }\n}"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 128,
    "completion_tokens": 43,
    "total_tokens": 171
  }
}

response from v1/chat/completions using the same question with mistral-nemo (works)

{
  "id": "chatcmpl-644",
  "object": "chat.completion",
  "created": 1725784981,
  "model": "mistral-nemo",
  "system_fingerprint": "fp_ollama",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "",
        "tool_calls": [
          {
            "id": "call_iaiz17nx",
            "type": "function",
            "function": {
              "name": "get_password",
              "arguments": "{}"
            }
          }
        ]
      },
      "finish_reason": "tool_calls"
    }
  ],
  "usage": {
    "prompt_tokens": 66,
    "completion_tokens": 16,
    "total_tokens": 82
  }
}

[sammcj]

You'll have to have a crack at it yourself, I'm not a maintainer for deepseek.