Why the same python tools run many times?

[KevinZhang19870314]

I am using Assistant with python tools, sometimes it will run many times for the same tools (see below log for tools start_time and end_time), and it also cause the tokens usage increase to double or more. So what is the reason? How can I fix it?

Here is the code and log:

--- code

import os
import pathlib

from phi.assistant import Assistant
from phi.llm.openai import OpenAIChat
from phi.tools.python import PythonTools
from pydantic import BaseModel, Field
from dotenv import load_dotenv

load_dotenv()

STORAGE_ROOT = os.getenv('STORAGE_ROOT')
PYTHON_TOOLS_STORAGE = os.getenv('PYTHON_TOOLS')
PYTHON_TOOLS_STORAGE_FOLDER_PATH = os.path.join(STORAGE_ROOT, PYTHON_TOOLS_STORAGE)

class TradesAPIParameter(BaseModel):
    startTime: str = Field(None, description="Start time for transaction creation. Use the python_tools tool to generate the code and get the value, in the format '2024-07-26T00:00:00+08:00'.")
    endTime: str = Field(None, description="End time for transaction creation. Use the python_tools tool to generate the code and get the value, in the format '2024-07-26T23:59:59+08:00'.")
    page: int = Field(1, description="Current page number for pagination.")
    size: int = Field(10, description="Number of records per page for pagination.")

movie_assistant = Assistant(
    # llm=OpenAIChat(model="gpt-3.5-turbo"),
    # llm=OpenAIChat(model="gpt-4o"),
    llm=OpenAIChat(model="gpt-4o-mini"),
    description="You are a parameter parser for the transaction record query API, responsible for parsing the natural language input by the user into the parameter fields required by this API.",
    output_model=TradesAPIParameter,
    tools=[PythonTools(base_dir=pathlib.Path(PYTHON_TOOLS_STORAGE_FOLDER_PATH))],
    show_tool_calls=False,
    debug_mode=False,
)

params = movie_assistant.run("Query the records of the last seven days, and 14 records are displayed per page.")
print("=========================")
print(params)
print(movie_assistant.llm.metrics)
print("=========================")

--- log

WARNING  PythonTools can run arbitrary code, please provide human supervision. 
INFO     Saved:                                                                
         D:\kevinzhang\storage\python_too
         ls\start_time.py                                                      
INFO     Running                                                               
         D:\kevinzhang\storage\python_too
         ls\start_time.py                                                      
INFO     Saved:                                                                
         D:\kevinzhang\storage\python_too
         ls\end_time.py                                                        
INFO     Running                                                               
         D:\kevinzhang\storage\python_too
         ls\end_time.py                                                        
INFO     Saved:                                                                
         D:\kevinzhang\storage\python_too
         ls\end_time.py                                                        
INFO     Running                                                               
         D:\kevinzhang\storage\python_too
         ls\end_time.py                                                        
INFO     Saved:                                                                
         D:\kevinzhang\storage\python_too
         ls\end_time.py                                                        
INFO     Running                                                               
         D:\kevinzhang\storage\python_too
         ls\end_time.py                                                        
INFO     Saved:                                                                
         D:\kevinzhang\storage\python_too
         ls\end_time.py                                                        
INFO     Running                                                               
         D:\kevinzhang\storage\python_too
         ls\end_time.py                                                        
INFO     Saved:                                                                
         D:\kevinzhang\storage\python_too
         ls\end_time.py                                                        
INFO     Running                                                               
         D:\kevinzhang\storage\python_too
         ls\end_time.py                                                        
INFO     Saved:                                                                
         D:\kevinzhang\storage\python_too
         ls\end_time.py                                                        
INFO     Running                                                               
         D:\kevinzhang\storage\python_too
         ls\end_time.py                                                        
INFO     Saved:                                                                
         D:\kevinzhang\storage\python_too
         ls\end_time.py                                                        
INFO     Running                                                               
         D:\kevinzhang\storage\python_too
         ls\end_time.py                                                        
INFO     Saved:                                                                
         D:\kevinzhang\storage\python_too
         ls\end_time.py                                                        
INFO     Running                                                               
         D:\kevinzhang\storage\python_too
         ls\end_time.py                                                        
INFO     Saved:                                                                
         D:\kevinzhang\storage\python_too
         ls\end_time.py                                                        
INFO     Running                                                               
         D:\kevinzhang\storage\python_too
         ls\end_time.py                                                        
=========================
{"startTime":"2024-07-24T09:29:24.197839+08:00","endTime":null,"page":1,"size":14}
{'response_times': [2.291413800092414, 1.480312200030312, 1.5049350999761373, 1.157371899811551, 1.6347784001845866, 1.32546820002608, 1.716063099913299, 1.7036544000729918, 1.2681862998288125, 2.304434100165963], 'prompt_tokens': 9046, 'completion_tokens': 546, 'total_tokens': 9592, 'tool_call_times': {'save_to_file_and_run': [0.018665699986740947, 0.010879899840801954, 0.005387699929997325, 0.004899699939414859, 0.004780299961566925, 0.0050095000769943, 0.005390499951317906, 0.005444800015538931, 0.005226599983870983, 0.011913999915122986]}}
=========================

And by the way, is my prompt not worked?

[ysolanky]

@KevinZhang19870314 , this appears to be a model limitation. Can you please share the output while using GPT 4o as the model?

[KevinZhang19870314]

@KevinZhang19870314 , this appears to be a model limitation. Can you please share the output while using GPT 4o as the model?

Thanks, GPT 4o works as normal, no python tools will be called more times.

So, I want to know the logic for different model while call the python tools. Can we control this logic, or what happened when call python tools many times?