Tools and Function Calls Not Working in OpenAILike Mode

[edisonzf2020]

The model inherently supports tools and function calls. These features work correctly in Ollama mode. However, when using OpenAILike mode, tools and function calls are not functioning as expected.

Steps to reproduce:

1. Use the model in Ollama mode - tools and function calls work properly.
2. Switch to OpenAILike mode.
3. Attempt to use tools or function calls - they fail to work.

Expected behavior: Tools and function calls should work in both Ollama and OpenAILike modes.

Actual behavior: Tools and function calls only work in Ollama mode, but not in OpenAILike mode.

Additional information: Model：Qwen2

Please investigate why tools and function calls are not operational in OpenAILike mode and provide a fix to ensure consistent functionality across both modes.

[ysolanky]

Hey @edisonzf2020! Currently, this behavior is expected because Qwen2 likely doesn't follow the OpenAI specification for function calling. When you use the OpenAILike class, it expects the model to adhere to the OpenAI specification. On the other hand, the Ollama class works correctly because it follows the Ollama function calling specification.

However, we're working on a new feature that should allow you to use OpenAILike with models like Qwen2 and still perform function calling :)

[KevinZhang19870314]

I am also met this problem when use OpenAILike class with models deepseek-coder, is this the same expected behavior? @ysolanky

[ysolanky]

@KevinZhang19870314 can you please share your code so that I can look into it?

[KevinZhang19870314]

@ysolanky Below is my code and related run logs for both gpt-4o-2024-08-06 and deepseek-chat model, you can comment out each for testing.

import time
from datetime import datetime

import pytz
from dotenv import load_dotenv
from phi.assistant import Assistant
from phi.llm.openai import OpenAIChat
from pydantic import BaseModel, Field

load_dotenv()

def retrieve_time(timezone: str = "Asia/Shanghai"):
    """
    Retrieve the current time in the specified timezone.

    Args:
            timezone (str): The timezone string (e.g., 'Asia/Shanghai').

    Returns:
            str: The formatted current time in the specified timezone.
    """
    local_timezone = pytz.timezone(timezone) 
    local_time = datetime.now(local_timezone)
    formatted_time = local_time.strftime('%Y-%m-%d %H:%M:%S %Z%z')
    return formatted_time

class TradesAPIParameter(BaseModel):
    startTime: str = Field(None, description="The start time for transaction creation. Make sure the value should be in the format '2024-07-26T00:00:00+08:00'. If no specific date or time frame is mentioned, do not set this value.")
    endTime: str = Field(None, description="The end time for transaction creation. Make sure the value should be in the format '2024-07-26T00:00:00+08:00'. If no specific date or time frame is mentioned, do not set this value.")
    insCode: str = Field(None, description="Institution code.")

start_time = time.time()

# open_ai_like_llm = OpenAILike(
#         model="deepseek-chat",
#         api_key=os.getenv("DEEPSEEK_API_KEY"),
#         base_url="https://api.deepseek.com",
#     )
assistant = Assistant(
    llm=OpenAIChat(model="gpt-4o-2024-08-06", temperature=0),
    # llm=open_ai_like_llm,
    description="You are a parameter parser for the transaction record query API, responsible for parsing the natural language input by the user into the parameter fields required by this API.",
    output_model=TradesAPIParameter,
    tools=[
        retrieve_time,
    ],
    instructions=[
        "Remove the null or None value fields from the final output.",
        "Ensure that both `startTime` and `endTime` are either both populated or both set to `None`.",
    ],
    add_chat_history_to_messages=False,
    add_chat_history_to_prompt=False,
    num_history_messages=0,
    create_memories=False,
    update_memory_after_run=False,
    show_tool_calls=True,
    debug_mode=False,
)

params = assistant.run("Please help me to query the today's transaction with institution code X123456")

end_time = time.time()
execution_time = end_time - start_time

print("=========================")
print(type(params))
if isinstance(params, TradesAPIParameter):
    print(params.model_dump())
else:
    try:
        import json
        print(json.loads(params))
    except Exception as e:
        print(params)

print(assistant.llm.metrics)
print(f"Execution Time: {execution_time} seconds")
print("=========================")

=============Logs for gpt-4o-2024-08-06================

=========================
<class 'str'>

 - Running: retrieve_time(timezone=Asia/Shanghai)

{
  "startTime": "2024-08-20T00:00:00+08:00",
  "endTime": "2024-08-20T23:59:59+08:00",
  "insCode": "X123456"
}
{'response_times': [1.0685460999957286, 1.6757817000034265], 'prompt_tokens': 930, 'completion_tokens': 75, 'total_tokens': 1005, 'tool_call_times': {'retrieve_time': [0.05491870001424104]}}
Execution Time: 2.814894676208496 seconds
=========================

=============Logs for deepseek-chat================

=========================
<class '__main__.TradesAPIParameter'>
{'startTime': None, 'endTime': None, 'insCode': 'X123456'}
{'response_times': [1.9781118999817409], 'prompt_tokens': 597, 'completion_tokens': 23, 'total_tokens': 620}
Execution Time: 1.9856033325195312 seconds
=========================