Open edisonzf2020 opened 1 month ago
Hey @edisonzf2020! Currently, this behavior is expected because Qwen2
likely doesn't follow the OpenAI
specification for function calling. When you use the OpenAILike
class, it expects the model to adhere to the OpenAI
specification. On the other hand, the Ollama
class works correctly because it follows the Ollama
function calling specification.
However, we're working on a new feature that should allow you to use OpenAILike
with models like Qwen2
and still perform function calling :)
I am also met this problem when use OpenAILike class with models deepseek-coder, is this the same expected behavior? @ysolanky
@KevinZhang19870314 can you please share your code so that I can look into it?
@ysolanky Below is my code and related run logs for both gpt-4o-2024-08-06 and deepseek-chat model, you can comment out each for testing.
import time
from datetime import datetime
import pytz
from dotenv import load_dotenv
from phi.assistant import Assistant
from phi.llm.openai import OpenAIChat
from pydantic import BaseModel, Field
load_dotenv()
def retrieve_time(timezone: str = "Asia/Shanghai"):
"""
Retrieve the current time in the specified timezone.
Args:
timezone (str): The timezone string (e.g., 'Asia/Shanghai').
Returns:
str: The formatted current time in the specified timezone.
"""
local_timezone = pytz.timezone(timezone)
local_time = datetime.now(local_timezone)
formatted_time = local_time.strftime('%Y-%m-%d %H:%M:%S %Z%z')
return formatted_time
class TradesAPIParameter(BaseModel):
startTime: str = Field(None, description="The start time for transaction creation. Make sure the value should be in the format '2024-07-26T00:00:00+08:00'. If no specific date or time frame is mentioned, do not set this value.")
endTime: str = Field(None, description="The end time for transaction creation. Make sure the value should be in the format '2024-07-26T00:00:00+08:00'. If no specific date or time frame is mentioned, do not set this value.")
insCode: str = Field(None, description="Institution code.")
start_time = time.time()
# open_ai_like_llm = OpenAILike(
# model="deepseek-chat",
# api_key=os.getenv("DEEPSEEK_API_KEY"),
# base_url="https://api.deepseek.com",
# )
assistant = Assistant(
llm=OpenAIChat(model="gpt-4o-2024-08-06", temperature=0),
# llm=open_ai_like_llm,
description="You are a parameter parser for the transaction record query API, responsible for parsing the natural language input by the user into the parameter fields required by this API.",
output_model=TradesAPIParameter,
tools=[
retrieve_time,
],
instructions=[
"Remove the null or None value fields from the final output.",
"Ensure that both `startTime` and `endTime` are either both populated or both set to `None`.",
],
add_chat_history_to_messages=False,
add_chat_history_to_prompt=False,
num_history_messages=0,
create_memories=False,
update_memory_after_run=False,
show_tool_calls=True,
debug_mode=False,
)
params = assistant.run("Please help me to query the today's transaction with institution code X123456")
end_time = time.time()
execution_time = end_time - start_time
print("=========================")
print(type(params))
if isinstance(params, TradesAPIParameter):
print(params.model_dump())
else:
try:
import json
print(json.loads(params))
except Exception as e:
print(params)
print(assistant.llm.metrics)
print(f"Execution Time: {execution_time} seconds")
print("=========================")
=============Logs for gpt-4o-2024-08-06================
=========================
<class 'str'>
- Running: retrieve_time(timezone=Asia/Shanghai)
{
"startTime": "2024-08-20T00:00:00+08:00",
"endTime": "2024-08-20T23:59:59+08:00",
"insCode": "X123456"
}
{'response_times': [1.0685460999957286, 1.6757817000034265], 'prompt_tokens': 930, 'completion_tokens': 75, 'total_tokens': 1005, 'tool_call_times': {'retrieve_time': [0.05491870001424104]}}
Execution Time: 2.814894676208496 seconds
=========================
=============Logs for deepseek-chat================
=========================
<class '__main__.TradesAPIParameter'>
{'startTime': None, 'endTime': None, 'insCode': 'X123456'}
{'response_times': [1.9781118999817409], 'prompt_tokens': 597, 'completion_tokens': 23, 'total_tokens': 620}
Execution Time: 1.9856033325195312 seconds
=========================
The model inherently supports tools and function calls. These features work correctly in Ollama mode. However, when using OpenAILike mode, tools and function calls are not functioning as expected.
Steps to reproduce:
Expected behavior: Tools and function calls should work in both Ollama and OpenAILike modes.
Actual behavior: Tools and function calls only work in Ollama mode, but not in OpenAILike mode.
Additional information: Model:Qwen2
Please investigate why tools and function calls are not operational in OpenAILike mode and provide a fix to ensure consistent functionality across both modes.