DDGS tool doesn't allow Ollama to handle rate limit correctly

[ofri]

Below is a snippet of the debug output from running cookbook/llms/ollama/tool_call.py after DDGS started rate limiting requests (from my IP?).

Supposedly, what is happening is:

• DDGS returns HTTP202 with indication of rate limit.
• Ollama.add_tool_call_error_message ( https://github.com/phidatahq/phidata/blob/c55547110638dd85158528c43d74006a77390a44/phi/llm/ollama/chat.py#L492 ) assumes any error is an arguments error and instructs the model to fix the arguments.
• The model has no ability to solve the actual issue, e.g., by adding a backoff delay.
• The model simply tries again (potentially getting stuck in an infinite loop?)

Potential approach for a solution: change the prompt to the model. Make sure enough info about the actual error is passed to the model. Ask the model to figure out if this is a parameters issue or not and to handle it accordingly. Suggest that if the model has no way to solve the issue (e.g., HTTP500) it should inform the user.

DEBUG    ============== user ==============
DEBUG    Whats happening in the US?
DEBUG    ============== assistant ==============
DEBUG    {
           "tool_calls": [
             {
               "name": "duckduckgo_news",
               "arguments": {
                 "query": "US news",
                 "max_results": 5
               }
             }
           ]
         }
DEBUG    Tool Calls: [
           {
             "type": "function",
             "function": {
               "name": "duckduckgo_news",
               "arguments": "{\"query\": \"US news\", \"max_results\": 5}"
             }
           }
         ]
DEBUG    ============== user ==============
DEBUG    https://duckduckgo.com/ 202 Ratelimit
DEBUG    ============== user ==============
DEBUG    Output from the tool indicates an arguments error, take a step back and adjust the tool arguments then use the same tool again
         with the new arguments. Ensure the response does not mention any failed tool calls, Just the adjusted tool calls.
DEBUG    ============== assistant ==============
DEBUG    {
           "tool_calls": [
             {
               "name": "duckduckgo_news",
               "arguments": {
                 "query": "",
                 "max_results": 5
               }
             },
             {
               "name": "duckduckgo_search",
               "arguments": {
                 "query": "US news",
                 "max_results": 5
               }
             }
           ]
         }

[jacobweiss2305]

Yeah you can add retry logic on top of the Assistant