progressEdd commented 3 months ago

Describe the issue

Hi all I'm trying to replicate the code in this tutorial, with a local llm using LM Studio endpoint https://microsoft.github.io/autogen/docs/tutorial/tool-use When I initiate the chat, open source models ignore the tools and instead opt to solve the calculate the result using order of operations.

I am able to replicate it using the AzureOpenAi GPT4 API.

Steps to reproduce

pure python

we expect the answer to be 221492.8

(44232 + 13312 / (232 - 32)) * 5

llama 3.1

code

I ran this code for llama 3.1

ollama31 = {
    "config_list": [
        {
            "model": "lmstudio-community/Meta-Llama-3.1-8B-Instruct-GGUF",
            "base_url": "http://localhost:1234/v1",
            "api_key": "lm-studio",
        },
    ],
    "cache_seed": None,  # Disable caching.
}
import os

from autogen import ConversableAgent

# Let's first define the assistant agent that suggests tool calls using gemma
assistant = ConversableAgent(
    name="Assistant",
    system_message="You are a helpful AI assistant. "
    "You can help with simple calculations."
    "Return 'TERMINATE' when the task is done.",
    llm_config=ollama31,
)

# The user proxy agent is used for interacting with the assistant agent
# and executes tool calls.
user_proxy = ConversableAgent(
    name="User",
    llm_config=False,
    is_termination_msg=lambda msg: msg.get("content") is not None and "TERMINATE" in msg["content"],
    human_input_mode="NEVER",
)

from typing import Annotated, Literal

Operator = Literal["+", "-", "*", "/"]

def calculator(a: int, b: int, operator: Annotated[Operator, "operator"]) -> int:
    print("Calculating", a, operator, b)
    if operator == "+":
        return a + b
    elif operator == "-":
        return a - b
    elif operator == "*":
        return a * b
    elif operator == "/":
        return int(a / b)
    else:
        raise ValueError("Invalid operator")

# Register the tool signature with the assistant agent.
assistant.register_for_llm(name="calculator", description="A simple calculator")(calculator)

# Register the tool function with the user proxy agent.
user_proxy.register_for_execution(name="calculator")(calculator)
from autogen import register_function

# Register the calculator function to the two agents.
register_function(
    calculator,
    caller=assistant,  # The assistant agent can suggest calls to the calculator.
    executor=user_proxy,  # The user proxy agent can execute the calculator calls.
    name="calculator",  # By default, the function name is used as the tool name.
    description="A simple calculator",  # A description of the tool.
)
import autogen
from autogen import AssistantAgent, UserProxyAgent
# Start logging using the agentchat_logging notebook
logging_session_id = autogen.runtime_logging.start(config={"dbname": "logs.db"})
print("Logging session ID: " + str(logging_session_id))

chat_result = user_proxy.initiate_chat(assistant, message="What is (44232 + 13312 / (232 - 32)) * 5?")

autogen.runtime_logging.stop()

output

it seemed to hallucinate in the add (44232 + 13312) and subtract (44232 + 66.56) operations

Logging session ID: 2631a5b5-ff5c-4b95-8e30-33b8af4269c5
User (to Assistant):

What is (44232 + 13312 [/](https://file+.vscode-resource.vscode-cdn.net/) (232 - 32)) * 5?

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
[autogen.oai.client: 07-23 18:08:46] {329} WARNING - Model lmstudio-community/Meta-Llama-3.1-8B-Instruct-GGUF/Meta-Llama-3.1-8B-Instruct-Q8_0.gguf is not found. The cost will be 0. In your config_list, add field {"price" : [prompt_price_per_1k, completion_token_price_per_1k]} for customized pricing.
Assistant (to User):

To evaluate this expression, we need to follow the order of operations (PEMDAS):

1. Subtract 32 from 232: 232 - 32 = 200
2. Divide 13312 by 200: 13312 [/](https://file+.vscode-resource.vscode-cdn.net/) 200 = 66.56
3. Add 44232 and 66.56: 44232 + 66.56 = 44398.56
4. Multiply the result by 5: 44398.56 * 5 = 221992.8

Therefore, (44232 + 13312 [/](https://file+.vscode-resource.vscode-cdn.net/) (232 - 32)) * 5 = 221992.8

--------------------------------------------------------------------------------
User (to Assistant):

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...

gemma 2

weirdly enough gemma didn't have issues doing the calculation.

code

gemma = {
    "config_list": [
        {
            "model": "lmstudio-community/gemma-2-9b-it-GGUF",
            "base_url": "http://localhost:1234/v1",
            "api_key": "lm-studio",
        },
    ],
    "cache_seed": None,  # Disable caching.
}

import os

from autogen import ConversableAgent

# Let's first define the assistant agent that suggests tool calls using gemma
assistant = ConversableAgent(
    name="Assistant",
    system_message="You are a helpful AI assistant. "
    "You can help with simple calculations. "
    "Return 'TERMINATE' when the task is done.",
    llm_config=gemma,
)

# The user proxy agent is used for interacting with the assistant agent
# and executes tool calls.
user_proxy = ConversableAgent(
    name="User",
    llm_config=False,
    is_termination_msg=lambda msg: msg.get("content") is not None and "TERMINATE" in msg["content"],
    human_input_mode="NEVER",
)

# Register the tool signature with the assistant agent.
assistant.register_for_llm(name="calculator", description="A simple calculator")(calculator)

# Register the tool function with the user proxy agent.
user_proxy.register_for_execution(name="calculator")(calculator)

from typing import Annotated, Literal

Operator = Literal["+", "-", "*", "/"]

def calculator(a: int, b: int, operator: Annotated[Operator, "operator"]) -> int:
    print("Calculating", a, operator, b)
    if operator == "+":
        return a + b
    elif operator == "-":
        return a - b
    elif operator == "*":
        return a * b
    elif operator == "/":
        return int(a / b)
    else:
        raise ValueError("Invalid operator")

from autogen import register_function

# Register the calculator function to the two agents.
register_function(
    calculator,
    caller=assistant,  # The assistant agent can suggest calls to the calculator.
    executor=user_proxy,  # The user proxy agent can execute the calculator calls.
    name="calculator",  # By default, the function name is used as the tool name.
    description="A simple calculator",  # A description of the tool.
)

import autogen
from autogen import AssistantAgent, UserProxyAgent
# Start logging using the agentchat_logging notebook
logging_session_id = autogen.runtime_logging.start(config={"dbname": "logs.db"})
print("Logging session ID: " + str(logging_session_id))

chat_result = user_proxy.initiate_chat(assistant, message="What is (44232 + 13312 / (232 - 32)) * 5?")

autogen.runtime_logging.stop()

output

Logging session ID: e170cbf8-a707-44e1-a24b-63f985455f81
User (to Assistant):

What is (44232 + 13312 [/](https://file+.vscode-resource.vscode-cdn.net/) (232 - 32)) * 5?

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
[autogen.oai.client: 07-16 15:24:35] {329} WARNING - Model lmstudio-community/gemma-2-9b-it-GGUF/gemma-2-9b-it-Q6_K.gguf is not found. The cost will be 0. In your config_list, add field {"price" : [prompt_price_per_1k, completion_token_price_per_1k]} for customized pricing.
Assistant (to User):

Here's how to solve that calculation:

1. **Parentheses first:** 232 - 32 = 200
2. **Division:** 13312 [/](https://file+.vscode-resource.vscode-cdn.net/) 200 = 66.56
3. **Addition:** 44232 + 66.56 = 44298.56
4. **Multiplication:** 44298.56 * 5 = 221492.8

**Answer:** 221492.8

TERMINATE

Screenshots and logs

lm studio config

llama3

{
  "name": "Exported from LM Studio on 7/23/2024, 7:07:38 PM",
  "load_params": {
    "n_ctx": 4000,
    "n_batch": 512,
    "rope_freq_base": 0,
    "rope_freq_scale": 0,
    "n_gpu_layers": 10,
    "use_mlock": true,
    "main_gpu": 0,
    "tensor_split": [
      0
    ],
    "seed": -1,
    "f16_kv": true,
    "use_mmap": true,
    "no_kv_offload": false,
    "num_experts_used": 0
  },
  "inference_params": {
    "n_threads": 4,
    "n_predict": -1,
    "top_k": 40,
    "min_p": 0.05,
    "top_p": 0.95,
    "temp": 0.8,
    "repeat_penalty": 1.1,
    "input_prefix": "<|eot_id|><|start_header_id|>user<|end_header_id|>\\n\\n",
    "input_suffix": "<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\n",
    "antiprompt": [
      "<|start_header_id|>",
      "<|eot_id|>"
    ],
    "pre_prompt": "You are a helpful, smart, kind, and efficient AI assistant. You always fulfill the user's requests to the best of your ability.",
    "pre_prompt_suffix": "",
    "pre_prompt_prefix": "<|start_header_id|>system<|end_header_id|>\\n\\n",
    "seed": -1,
    "tfs_z": 1,
    "typical_p": 1,
    "repeat_last_n": 64,
    "frequency_penalty": 0,
    "presence_penalty": 0,
    "n_keep": 0,
    "logit_bias": {},
    "mirostat": 0,
    "mirostat_tau": 5,
    "mirostat_eta": 0.1,
    "memory_f16": true,
    "multiline_input": false,
    "penalize_nl": true
  }
}

gemma 2 config

{
  "name": "Exported from LM Studio on 7/23/2024, 7:09:40 PM",
  "load_params": {
    "n_ctx": 4000,
    "n_batch": 512,
    "rope_freq_base": 0,
    "rope_freq_scale": 0,
    "n_gpu_layers": 10,
    "use_mlock": true,
    "main_gpu": 0,
    "tensor_split": [
      0
    ],
    "seed": -1,
    "f16_kv": true,
    "use_mmap": true,
    "no_kv_offload": false,
    "num_experts_used": 0
  },
  "inference_params": {
    "n_threads": 4,
    "n_predict": -1,
    "top_k": 40,
    "min_p": 0.05,
    "top_p": 0.95,
    "temp": 0.8,
    "repeat_penalty": 1.1,
    "input_prefix": "<start_of_turn>user\\n",
    "input_suffix": "<end_of_turn>\\n<start_of_turn>model\\n",
    "antiprompt": [
      "<start_of_turn>user",
      "<start_of_turn>model",
      "<end_of_turn>"
    ],
    "pre_prompt": "",
    "pre_prompt_suffix": "",
    "pre_prompt_prefix": "",
    "seed": -1,
    "tfs_z": 1,
    "typical_p": 1,
    "repeat_last_n": 64,
    "frequency_penalty": 0,
    "presence_penalty": 0,
    "n_keep": 0,
    "logit_bias": {},
    "mirostat": 0,
    "mirostat_tau": 5,
    "mirostat_eta": 0.1,
    "memory_f16": true,
    "multiline_input": false,
    "penalize_nl": true
  }
}

Additional Information

poetry pyproject dependencies

[tool.poetry.dependencies]
python = ">=3.10,<3.13"
pandas = "*"
streamlit = "^1.31.0"
langchain = "^0.1.5"
python-dotenv = "^1.0.1"
emoji = "^2.10.1"
async-timeout = "^4.0.3"
numpy = "*"
Pillow = "*"
openai = "*"
ipykernel = "^6.29.2"
scikit-learn = "^1.4.0"
ipywidgets = "^8.1.2"
openpyxl = "*"
kaleido = "0.2.1"
langchain-community = "^0.0.21"
langchain-openai = "^0.0.6"
faker = "^24.0.0"
langchain-experimental = "^0.0.52"
tabulate = "^0.9.0"
geopy = "^2.4.1"
panel = "^1.3.8"
jupyter-bokeh = "^4.0.1"
pyviz-comms = "^3.0.2"
nltk = "^3.8.1"
lxml = "^5.2.1"
fuzzywuzzy = "^0.18.0"
python-levenshtein = "^0.25.1"
googlemaps = "^4.10.0"
pyautogen = {extras = ["long-context"], version = "^0.2.31"}
datasets = "^2.20.0"

marklysze commented 3 months ago

Hi @progressEdd, LM Studio doesn't yet support function calling, unfortunately.

We're currently working on an Ollama client class that supports function calling, see PR #3056. If you're able to use Ollama you could try that class.

Alternatively, if you are able to use LiteLLM + Ollama, that supports function calling using the current AutoGen code base, I have an updated documentation PR, here #3197.

If you're able to try those let me know. Llama 3 and 3.1 work well for function calling with both these options.

progressEdd commented 3 months ago

Hi @progressEdd, LM Studio doesn't yet support function calling, unfortunately.

We're currently working on an Ollama client class that supports function calling, see PR #3056. If you're able to use Ollama you could try that class.

Alternatively, if you are able to use LiteLLM + Ollama, that supports function calling using the current AutoGen code base, I have an updated documentation PR, here #3197.

If you're able to try those let me know. Llama 3 and 3.1 work well for function calling with both these options.

Gotcha, I'll try those and come back if I have issues

progressEdd commented 3 months ago

Hi @progressEdd, LM Studio doesn't yet support function calling, unfortunately.

We're currently working on an Ollama client class that supports function calling, see PR #3056. If you're able to use Ollama you could try that class.

Alternatively, if you are able to use LiteLLM + Ollama, that supports function calling using the current AutoGen code base, I have an updated documentation PR, here #3197.

If you're able to try those let me know. Llama 3 and 3.1 work well for function calling with both these options.

I tried the #3197, I am able to get the llama3.1, llama3instruct, and gemma to use tools! When I try all 3 models they seem to hallucinate. My suspicion is that ollama pulled lower quantized models since I'm running it locally on a 32gb M2 macbook pro. When I used the a models >Q5 in lmstudio (see the OP) the llms was able to reason with order of operations. The hallucination only occurred when selecting the numbers for subsequent operations.

If I were to tweak the calculator descriptions or the system prompt, I could potentially see better results. I can rerun these tests once the lmstudio and ollama function calling changes get merged.

Model results

summary

for this use case of doing math with tools:
- llama models (3.1 and 3instruct): don't appear to understand the input for the functions, as the llm will try to put all numbers as inputs into the function
- gemma2: follows the instructions for the functions on the first step, but doesn't know what to do with the result.
- Mixtral: did not complete, empty outputs.
  - I suspect it wasn't fine tuned for functions (model is 6 months old)

shared code

import os

from autogen import ConversableAgent

# Let's first define the assistant agent that suggests tool calls using gemma
llama31LiteLlm = {
    "config_list": [
        {
            "model": "NotRequired",  # Loaded with LiteLLM command
            "api_key": "NotRequired",  # Not needed
            "base_url": "http://0.0.0.0:4000",  # Your LiteLLM URL
            "price": [0, 0],  # Put in price per 1K tokens [prompt, response] as free!
        }
    ],
    "cache_seed": None,  # Turns off caching, useful for testing different models
}

# Let's first define the assistant agent that suggests tool calls using gemma
assistant = ConversableAgent(
    name="Assistant",
    system_message="You are a helpful AI assistant. "
    "You can help with simple calculations."
    "Return 'TERMINATE' when the task is done.",
    llm_config=llama31LiteLlm, # variable name will change for different models for me to keep track
)

from typing import Annotated, Literal

Operator = Literal["+", "-", "*", "/"]

def calculator(a: int, b: int, operator: Annotated[Operator, "operator"]) -> int:
    print("Calculating", a, operator, b)
    if operator == "+":
        return a + b
    elif operator == "-":
        return a - b
    elif operator == "*":
        return a * b
    elif operator == "/":
        return int(a / b)
    else:
        raise ValueError("Invalid operator")

# Register the tool signature with the assistant agent.
assistant.register_for_llm(name="calculator", description="A simple calculator")(calculator)

# Register the tool function with the user proxy agent.
user_proxy.register_for_execution(name="calculator")(calculator)
from autogen import register_function

# Register the calculator function to the two agents.
register_function(
    calculator,
    caller=assistant,  # The assistant agent can suggest calls to the calculator.
    executor=user_proxy,  # The user proxy agent can execute the calculator calls.
    name="calculator",  # By default, the function name is used as the tool name.
    description="A simple calculator",  # A description of the tool.
)
import autogen
from autogen import AssistantAgent, UserProxyAgent
# Start logging using the agentchat_logging notebook
logging_session_id = autogen.runtime_logging.start(config={"dbname": "logs.db"})
print("Logging session ID: " + str(logging_session_id))

chat_result = user_proxy.initiate_chat(assistant, message="What is (44232 + 13312 / (232 - 32)) * 5?")

autogen.runtime_logging.stop()

Llama3.1

the base llama3.1 model seems to try to put every number into the function

output

Logging session ID: 14cd6953-315d-40ad-acd5-e0fb45cf9878
User (to Assistant):

What is (44232 + 13312 [/](https://file+.vscode-resource.vscode-cdn.net/) (232 - 32)) * 5?

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_a0f7c234-43ec-40c5-a422-db1ea56e8b25): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "+", "c": 232, "d": 32}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_a0f7c234-43ec-40c5-a422-db1ea56e8b25) *****
Error: calculator() got an unexpected keyword argument 'c'
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_3a63b1c2-8748-45e8-8a1f-63db036872a0): calculator *****
Arguments: 
{"argument_name": "operator", "argument_value": "+"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_3a63b1c2-8748-45e8-8a1f-63db036872a0) *****
Error: calculator() got an unexpected keyword argument 'argument_name'
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_e3beff58-5faa-4d51-b01c-1f4bbdd9042f): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_e3beff58-5faa-4d51-b01c-1f4bbdd9042f) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_c937a7fe-81eb-4466-821c-4ca567579635): calculator *****
Arguments: 
{"a": 232, "b": 32, "operator": "-"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 232 - 32
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_c937a7fe-81eb-4466-821c-4ca567579635) *****
200
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_3c4a9510-4024-43b0-9014-95b0a431f7fc): calculator *****
Arguments: 
{"a": 44232, "b": 200, "operator": "+"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 + 200
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_3c4a9510-4024-43b0-9014-95b0a431f7fc) *****
44432
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_8b5fa649-4796-48ff-b718-1e910fba52af): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_8b5fa649-4796-48ff-b718-1e910fba52af) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_92e00e80-9b13-44c0-81b6-6e16f3f74794): calculator *****
Arguments: 
{"a": 200, "b": 5, "operator": "*"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 200 * 5
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_92e00e80-9b13-44c0-81b6-6e16f3f74794) *****
1000
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_b11c3411-fdf7-4ede-b3c5-2c75b9d9a1c7): calculator *****
Arguments: 
{"a": 44432, "b": 3, "operator": "*"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44432 * 3
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_b11c3411-fdf7-4ede-b3c5-2c75b9d9a1c7) *****
133296
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_79576f7d-1361-4613-a2bf-4fcff4917a61): calculator *****
Arguments: 
{"a": 44232, "b": 200, "operator": "+"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 + 200
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_79576f7d-1361-4613-a2bf-4fcff4917a61) *****
44432
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_e75c4567-24b0-48e7-ac96-4429e5fedd78): calculator *****
Arguments: 
{"a": 133296, "b": 3, "operator": "*"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 133296 * 3
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_e75c4567-24b0-48e7-ac96-4429e5fedd78) *****
399888
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_9d3ff73f-889d-4f2e-b9cf-853e1aa74cad): TERMINATE *****
Arguments: 
{"argument_name": "result", "argument_value": 1995556}
**************************************************************************************

--------------------------------------------------------------------------------
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_9d3ff73f-889d-4f2e-b9cf-853e1aa74cad) *****
Error: Function TERMINATE not found.
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_aa50efd9-4e10-443f-ad26-1dde8a666d20): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "+", "c": 232, "d": 32}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_aa50efd9-4e10-443f-ad26-1dde8a666d20) *****
Error: calculator() got an unexpected keyword argument 'c'
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_75dab496-f242-4a21-84f8-fb91dd91f1ef): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "+"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 + 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_75dab496-f242-4a21-84f8-fb91dd91f1ef) *****
57544
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_2e8b3ef5-a204-4829-9e80-ce925d464fbd): calculator *****
Arguments: 
{"argument_name": "result", "argument_value": 57544}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_2e8b3ef5-a204-4829-9e80-ce925d464fbd) *****
Error: calculator() got an unexpected keyword argument 'argument_name'
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_25b1f57e-ce89-4959-8aaf-2fcf618c7e19): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "+", "c": 232, "d": 32}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_25b1f57e-ce89-4959-8aaf-2fcf618c7e19) *****
Error: calculator() got an unexpected keyword argument 'c'
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_a2222368-3624-4acf-a875-b24a44d39f43): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "+", "c": 232, "d": 32}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_a2222368-3624-4acf-a875-b24a44d39f43) *****
Error: calculator() got an unexpected keyword argument 'c'
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_79f4c5d9-4937-48d0-8e70-7f5de5c29421): TERMINATE *****
Arguments: 
{"argument_name": "result", "argument_value": 57544}
**************************************************************************************

--------------------------------------------------------------------------------
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_79f4c5d9-4937-48d0-8e70-7f5de5c29421) *****
Error: Function TERMINATE not found.
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_2a2d46d7-c442-4444-a2be-ee61b43922f3): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "+"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 + 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_2a2d46d7-c442-4444-a2be-ee61b43922f3) *****
57544
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...

Llama3:instruct

Logging session ID: 6fb5abfd-73a4-4eb0-9f4e-0feb5965bf87
User (to Assistant):

What is (44232 + 13312 [/](https://file+.vscode-resource.vscode-cdn.net/) (232 - 32)) * 5?

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_8a81b639-600a-456a-93ee-0c04229ae42b): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_8a81b639-600a-456a-93ee-0c04229ae42b) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_a586870b-0021-499b-9a5f-047714059945): calculator *****
Arguments: 
{"a": 44232, "b": 13312.0, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312.0
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_a586870b-0021-499b-9a5f-047714059945) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_a1f57def-cadb-43d7-863c-e2ce6d3f27d8): calculator *****
Arguments: 
{"a": 44232, "b": 13312.0, "operator": "*"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 * 13312.0
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_a1f57def-cadb-43d7-863c-e2ce6d3f27d8) *****
588816384.0
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_844d4d97-b3e3-4841-a2a2-484e947355cd): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_844d4d97-b3e3-4841-a2a2-484e947355cd) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_bb464add-be86-4a5e-88e1-6ce5004280f3): calculator *****
Arguments: 
{"a": 44232, "b": 13312.0, "operator": "*"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 * 13312.0
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_bb464add-be86-4a5e-88e1-6ce5004280f3) *****
588816384.0
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_b1ae0fb0-7210-4428-9924-a4d0d716e9df): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_b1ae0fb0-7210-4428-9924-a4d0d716e9df) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_a99a70c0-12b8-450a-bfdf-66e8e2940fbe): calculator *****
Arguments: 
{"a": 44232, "b": 13312.0, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312.0
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_a99a70c0-12b8-450a-bfdf-66e8e2940fbe) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_45c5cf8f-951a-4be7-94a5-b7a45717cd8e): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_45c5cf8f-951a-4be7-94a5-b7a45717cd8e) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_abf9c1d8-a994-4369-bd47-fb59406fa30a): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_abf9c1d8-a994-4369-bd47-fb59406fa30a) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_95877ac3-f71d-4bcf-935d-1bed89f08b65): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_95877ac3-f71d-4bcf-935d-1bed89f08b65) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_f7b4e960-ccd1-46b2-b959-c2309813ece9): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "*"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 * 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_f7b4e960-ccd1-46b2-b959-c2309813ece9) *****
588816384
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_d8729656-85c5-4a79-8a77-5796014c2047): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_d8729656-85c5-4a79-8a77-5796014c2047) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_6cc7c939-789d-4d0f-a686-a71e79357a08): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_6cc7c939-789d-4d0f-a686-a71e79357a08) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_89600e41-0702-4b97-955a-fede20dcb62b): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_89600e41-0702-4b97-955a-fede20dcb62b) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_1efb3b3c-2c1f-424d-9712-58e327a18931): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_1efb3b3c-2c1f-424d-9712-58e327a18931) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_18845096-b403-44a6-9665-1677a61f0bf6): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_18845096-b403-44a6-9665-1677a61f0bf6) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_e5bd3577-17f8-4897-8501-3b4698cba16d): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_e5bd3577-17f8-4897-8501-3b4698cba16d) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_13cfcc7f-f292-4db7-aaa5-4476f988b934): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/", "c": 200, "d": -32, "e": 5}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_13cfcc7f-f292-4db7-aaa5-4476f988b934) *****
Error: calculator() got an unexpected keyword argument 'c'
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_2191cce6-567a-4ffc-b21f-b37355a503a7): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_2191cce6-567a-4ffc-b21f-b37355a503a7) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_996c6389-6166-4667-8970-d8175e122ae2): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_996c6389-6166-4667-8970-d8175e122ae2) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_0a4d25c8-ed18-4d36-ac3a-fdc1cc87d9eb): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_0a4d25c8-ed18-4d36-ac3a-fdc1cc87d9eb) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_06c4811c-4b38-4d00-a81f-04f8b950dea2): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "+"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 + 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_06c4811c-4b38-4d00-a81f-04f8b950dea2) *****
57544
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_d4bba4bf-7aad-4285-83a6-58fa3cc4120c): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_d4bba4bf-7aad-4285-83a6-58fa3cc4120c) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_a923e9b9-d425-475f-b349-097b610de506): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_a923e9b9-d425-475f-b349-097b610de506) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_5ccf0311-9849-4f7a-bc4e-bdaad64f36b5): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_5ccf0311-9849-4f7a-bc4e-bdaad64f36b5) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_1799233e-2f19-49f2-98b6-6e463105f160): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_1799233e-2f19-49f2-98b6-6e463105f160) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_a1c22475-b073-4468-9601-1a656406ae3d): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_a1c22475-b073-4468-9601-1a656406ae3d) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_73b98626-4e71-452c-b41e-77e5f7a829b8): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_73b98626-4e71-452c-b41e-77e5f7a829b8) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_7a80d63d-a80d-4910-9444-e283481dcbc0): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_7a80d63d-a80d-4910-9444-e283481dcbc0) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_0ac923bc-4a7f-490a-9f47-7ad34a70c62e): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_0ac923bc-4a7f-490a-9f47-7ad34a70c62e) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_c80bbb63-7d76-411d-aa2d-c8c8382da40c): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_c80bbb63-7d76-411d-aa2d-c8c8382da40c) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_9cd4ec68-460e-49bc-95c9-7673959519f7): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_9cd4ec68-460e-49bc-95c9-7673959519f7) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_03d7557d-9d4d-4774-a8c5-680138c2b9b7): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_03d7557d-9d4d-4774-a8c5-680138c2b9b7) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_aa23c857-a55a-495a-9dd2-d88f05a37804): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_aa23c857-a55a-495a-9dd2-d88f05a37804) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_59cae8a6-4139-49b4-a5a6-fb592de5fc00): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_59cae8a6-4139-49b4-a5a6-fb592de5fc00) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_9fe40466-bc28-453b-81d4-6e026af8b46d): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_9fe40466-bc28-453b-81d4-6e026af8b46d) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_6e66d18e-caf4-410c-b7c3-4c6da5a31b36): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_6e66d18e-caf4-410c-b7c3-4c6da5a31b36) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_68314522-5d5a-42c4-bda7-86f3c50fd241): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_68314522-5d5a-42c4-bda7-86f3c50fd241) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_6f9620f1-4932-4b7a-926f-4a053c54e20e): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_6f9620f1-4932-4b7a-926f-4a053c54e20e) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_f44304c0-902b-4027-934b-54fb0adbd839): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "*"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 * 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_f44304c0-902b-4027-934b-54fb0adbd839) *****
588816384
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_890eb242-dd91-41e3-8d6a-e92a4df9d672): calculator *****
Arguments: 
{"a": 44232, "b": 13312.0, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312.0
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_890eb242-dd91-41e3-8d6a-e92a4df9d672) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_4f7566b2-3eeb-4ceb-9955-c99525aab54b): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_4f7566b2-3eeb-4ceb-9955-c99525aab54b) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_89f23dce-2147-4cd8-8ba0-c42bc35d8311): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_89f23dce-2147-4cd8-8ba0-c42bc35d8311) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_7031efb8-ed63-46eb-8809-d90165d4ced3): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_7031efb8-ed63-46eb-8809-d90165d4ced3) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_17c81ef9-1f16-43fd-8ea9-8cd1d92a7e6e): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_17c81ef9-1f16-43fd-8ea9-8cd1d92a7e6e) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_3c132f79-bd83-4c1d-a6b5-43bf24bdd16a): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "*"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 * 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_3c132f79-bd83-4c1d-a6b5-43bf24bdd16a) *****
588816384
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_c705c6f1-19a7-45d7-afa5-1cb03e536cf4): calculator *****
Arguments: 
{"a": 44232, "b": 13312.0, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312.0
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_c705c6f1-19a7-45d7-afa5-1cb03e536cf4) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_7cea8dbe-24ef-4c49-b559-a76234380f91): calculator *****
Arguments: 
{"a": 44232, "b": 13312.0, "operator": "/", "c": 232}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_7cea8dbe-24ef-4c49-b559-a76234380f91) *****
Error: calculator() got an unexpected keyword argument 'c'
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_87c99b2f-5ee3-4be6-87ac-47ecbf25f5c9): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "+", "c": null}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_87c99b2f-5ee3-4be6-87ac-47ecbf25f5c9) *****
Error: calculator() got an unexpected keyword argument 'c'
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_589e43bc-9b46-493a-9849-c9a65cbd5f12): calculator *****
Arguments: 
{"a": 44232, "b": 13312.0, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312.0
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_589e43bc-9b46-493a-9849-c9a65cbd5f12) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_9e12bed2-1e34-4087-832f-bd4e7362fb5d): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_9e12bed2-1e34-4087-832f-bd4e7362fb5d) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_9dbe5f8a-45d9-4c1d-8fc9-44afbd01a345): calculator *****
Arguments: 
{"a": 44232, "b": 13312.0, "operator": "*"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 * 13312.0
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_9dbe5f8a-45d9-4c1d-8fc9-44afbd01a345) *****
588816384.0
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_bc6411d8-083a-4bbb-b643-f432aba594b7): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_bc6411d8-083a-4bbb-b643-f432aba594b7) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_5c54a515-591e-43e1-a597-cd91156045ea): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_5c54a515-591e-43e1-a597-cd91156045ea) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_ca83c8cf-717b-4233-836b-94616cec741d): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_ca83c8cf-717b-4233-836b-94616cec741d) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_87f4cb9b-27f7-4728-b7e4-fda7ae1750ca): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_87f4cb9b-27f7-4728-b7e4-fda7ae1750ca) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_2866242e-6cbd-4db9-8713-204acd26be8b): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_2866242e-6cbd-4db9-8713-204acd26be8b) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_97a64c93-69ec-4c06-9df4-81d56e79802e): calculator *****
Arguments: 
{"a": 44232, "b": 13312.0, "operator": "+", "c": null}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_97a64c93-69ec-4c06-9df4-81d56e79802e) *****
Error: calculator() got an unexpected keyword argument 'c'
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_894eeac2-a56f-4ab2-bc18-be674a123ae4): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_894eeac2-a56f-4ab2-bc18-be674a123ae4) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_6bbfa59f-9d14-47bd-98c1-e97df2725f83): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_6bbfa59f-9d14-47bd-98c1-e97df2725f83) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_3c5a2a87-2421-48fa-8d48-9fbce20ac80d): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_3c5a2a87-2421-48fa-8d48-9fbce20ac80d) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_2bb2d457-0185-4eaa-86a4-2551a81ffd00): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_2bb2d457-0185-4eaa-86a4-2551a81ffd00) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_80296f94-19cb-47bd-8e76-efbe122f1d68): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_80296f94-19cb-47bd-8e76-efbe122f1d68) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_2bed958e-afef-4cff-9485-446fb9f6eb33): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_2bed958e-afef-4cff-9485-446fb9f6eb33) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_c078f78a-b43c-41a8-9b19-70ef9f47976a): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_c078f78a-b43c-41a8-9b19-70ef9f47976a) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_0439e304-dfe1-4c3c-b8eb-6903afec1658): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": ["+", "/"], "c": null}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_0439e304-dfe1-4c3c-b8eb-6903afec1658) *****
Error: calculator() got an unexpected keyword argument 'c'
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_596a7bc5-e79e-4769-abca-0bd6ef647cbb): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_596a7bc5-e79e-4769-abca-0bd6ef647cbb) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_dfbd905d-1b9d-4455-a68d-174a06df3b75): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_dfbd905d-1b9d-4455-a68d-174a06df3b75) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_4a2b868d-6ee5-46b4-81f0-f3d28914eb0f): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "+", "*": 5}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_4a2b868d-6ee5-46b4-81f0-f3d28914eb0f) *****
Error: calculator() got an unexpected keyword argument '*'
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_594653ea-e67d-442e-a05e-81a2359368a9): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "+", "*": 5}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_594653ea-e67d-442e-a05e-81a2359368a9) *****
Error: calculator() got an unexpected keyword argument '*'
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_049bddd7-0344-48ff-bbde-a3d3c0132880): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "*"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 * 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_049bddd7-0344-48ff-bbde-a3d3c0132880) *****
588816384
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_e0877e24-ab72-42d5-b4a5-30ef7eaa4e6b): calculator *****
Arguments: 
{"a": 44232, "b": 13312.0, "operator": "*"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 * 13312.0
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_e0877e24-ab72-42d5-b4a5-30ef7eaa4e6b) *****
588816384.0
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_1e030bb6-e61f-4861-98b1-583ff6eaedb0): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_1e030bb6-e61f-4861-98b1-583ff6eaedb0) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_c073184d-9d74-4586-a81d-58be17c9ecf3): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_c073184d-9d74-4586-a81d-58be17c9ecf3) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_6200820e-96c5-445b-ba53-40477e471a60): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_6200820e-96c5-445b-ba53-40477e471a60) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_6689c2fb-4d5b-4451-906c-4fea3613f002): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "*"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 * 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_6689c2fb-4d5b-4451-906c-4fea3613f002) *****
588816384
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_be0d18e1-daa6-44d1-aa43-0f291be9ea71): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "*"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 * 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_be0d18e1-daa6-44d1-aa43-0f291be9ea71) *****
588816384
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_b9a14a04-a48a-482d-adb4-7e41d7a9ca05): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": ["+", "/"], "c": null}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_b9a14a04-a48a-482d-adb4-7e41d7a9ca05) *****
Error: calculator() got an unexpected keyword argument 'c'
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_9dd792bc-21d2-466f-9177-1aa059dff688): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_9dd792bc-21d2-466f-9177-1aa059dff688) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_8f3f8ce5-2d89-469b-931d-4400d96896e9): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "*"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 * 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_8f3f8ce5-2d89-469b-931d-4400d96896e9) *****
588816384
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_a02f8617-8a81-4e32-8e4a-afa0c42a708e): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 [/](https://file+.vscode-resource.vscode-cdn.net/) 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_a02f8617-8a81-4e32-8e4a-afa0c42a708e) *****
3
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_5df403c1-66d0-4cd0-9aa8-0490fe2f5fe8): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "+", "c": "5"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_5df403c1-66d0-4cd0-9aa8-0490fe2f5fe8) *****
Error: calculator() got an unexpected keyword argument 'c'
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_b01d1875-2e21-441b-9779-03d9d850699c): calculator *****
Arguments: 
{"a": 44232, "b": 13312.0, "operator": "*"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 * 13312.0
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_b01d1875-2e21-441b-9779-03d9d850699c) *****
588816384.0
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...

gemma 2

Logging session ID: 0e2d4663-55c6-4869-8700-f4dcc7df0bbb
User (to Assistant):

What is (44232 + 13312 [/](https://file+.vscode-resource.vscode-cdn.net/) (232 - 32)) * 5?

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_caa39b43-9c27-40c7-8cd2-8b956ac594c4): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "+"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 + 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_caa39b43-9c27-40c7-8cd2-8b956ac594c4) *****
57544
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_e89b5161-bf37-4ac6-b11a-30fcb1040632): calculator *****
Arguments: 
{"a": 57544, "b": 0, "operator": "*"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 57544 * 0
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_e89b5161-bf37-4ac6-b11a-30fcb1040632) *****
0
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_f938d5ab-2edb-4103-9245-7fa189b781c7): calculator *****
Arguments: 
{"a": 0, "b": 0, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 0 [/](https://file+.vscode-resource.vscode-cdn.net/) 0
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_f938d5ab-2edb-4103-9245-7fa189b781c7) *****
Error: division by zero
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_81f7b185-7431-4f28-ba7c-c84145a48152): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "+"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 + 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_81f7b185-7431-4f28-ba7c-c84145a48152) *****
57544
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_a5cd79b7-27f8-4a4b-8562-559d720a89a2): calculator *****
Arguments: 
{"a": 57544, "b": 0, "operator": "*"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 57544 * 0
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_a5cd79b7-27f8-4a4b-8562-559d720a89a2) *****
0
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_647f1aca-7bc0-457a-bdf0-0f83a701936f): calculator *****
Arguments: 
{"a": 0, "b": 0, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 0 [/](https://file+.vscode-resource.vscode-cdn.net/) 0
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_647f1aca-7bc0-457a-bdf0-0f83a701936f) *****
Error: division by zero
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_9fea4af5-7e40-463a-8af6-2294990c3a07): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "+"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 + 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_9fea4af5-7e40-463a-8af6-2294990c3a07) *****
57544
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_eee31bd6-e677-416f-bf78-69230bc0db77): calculator *****
Arguments: 
{"a": 57544, "b": 0, "operator": "*"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 57544 * 0
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_eee31bd6-e677-416f-bf78-69230bc0db77) *****
0
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_c3cb534e-370f-456e-9ae0-48a1fc4bb22e): calculator *****
Arguments: 
{"a": 0, "b": 0, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 0 [/](https://file+.vscode-resource.vscode-cdn.net/) 0
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_c3cb534e-370f-456e-9ae0-48a1fc4bb22e) *****
Error: division by zero
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_d8f04dbd-62ab-4298-a5e9-8949da425c08): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "+"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 + 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_d8f04dbd-62ab-4298-a5e9-8949da425c08) *****
57544
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_29d97e2e-654f-4e73-a3ca-4bc4733c19b5): calculator *****
Arguments: 
{"a": 57544, "b": 0, "operator": "*"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 57544 * 0
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_29d97e2e-654f-4e73-a3ca-4bc4733c19b5) *****
0
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_56bca7e5-bb29-4ec0-94d7-7483c31ea643): calculator *****
Arguments: 
{"a": 0, "b": 0, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 0 [/](https://file+.vscode-resource.vscode-cdn.net/) 0
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_56bca7e5-bb29-4ec0-94d7-7483c31ea643) *****
Error: division by zero
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_98e9e3a8-6e5f-4377-93c9-d3455cceba55): calculator *****
Arguments: 
{"a": 44232, "b": 13312, "operator": "+"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 44232 + 13312
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_98e9e3a8-6e5f-4377-93c9-d3455cceba55) *****
57544
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_bc2b8af3-0d6b-4212-89f5-fd2d772696c4): calculator *****
Arguments: 
{"a": 57544, "b": 0, "operator": "*"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 57544 * 0
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_bc2b8af3-0d6b-4212-89f5-fd2d772696c4) *****
0
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Assistant (to User):

***** Suggested tool call (call_937cfdea-af05-4430-873b-19ecd7f573dc): calculator *****
Arguments: 
{"a": 0, "b": 0, "operator": "/"}
***************************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION calculator...
Calculating 0 [/](https://file+.vscode-resource.vscode-cdn.net/) 0
User (to Assistant):

User (to Assistant):

***** Response from calling tool (call_937cfdea-af05-4430-873b-19ecd7f573dc) *****
Error: division by zero
**********************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...

progressEdd commented 3 months ago

My suspicion is that ollama pulled lower quantized models since I'm running it locally on a 32gb M2 macbook pro. When I used the a models >Q5 in lmstudio (see the OP) the llms was able to reason with order of operations. The hallucination only occurred when selecting the numbers for subsequent operations.

I was correct, I didn't browse the ollama models page as closely pulling the 8b instruct q6 model. will check if results are better

@marklysze now that ollama has tool support, https://ollama.com/blog/tool-support Will autogen work out of the box? Or will I need to wait for an update?

marklysze commented 3 months ago

@marklysze now that ollama has tool support, https://ollama.com/blog/tool-support Will autogen work out of the box? Or will I need to wait for an update?

Hey @progressEdd, I've just updated the Ollama client class PR #3056 to support native Ollama tool calling. The tool calling works okay using this code - but like LiteLLM+Ollama it still suffers with calling a tool over and over - for which we've implemented a hide_tools configuration property. Check out that PR if you want to try it out :).

I don't think it will work out of the box but you could try setting the base_url to Ollama, just make sure you have version 0.3.0+ of Ollama.

remidiy commented 2 weeks ago

@marklysze Does the function/tool calls work for the custom models which follow the OpenAI API endpoints contracts but behind a proxy server?

marklysze commented 2 weeks ago

@marklysze Does the function/tool calls work for the custom models which follow the OpenAI API endpoints contracts but behind a proxy server?

Hey @remidiy, just to clarify are you able to provide an example of the llm_config(s) (without API keys) so I can understand how the proxy is involved?

If the configs are using api_type=ollama or gemini or anthropic, etc. then it should use those client classes and they all support function calling (though not always when streaming).

ekzhu commented 2 weeks ago

Closing this as the op question was answered

remidiy commented 2 weeks ago

@marklysze Does the function/tool calls work for the custom models which follow the OpenAI API endpoints contracts but behind a proxy server?

Hey @remidiy, just to clarify are you able to provide an example of the llm_config(s) (without API keys) so I can understand how the proxy is involved?

If the configs are using api_type=ollama or gemini or anthropic, etc. then it should use those client classes and they all support function calling (though not always when streaming).

@marklysze Sorry I can't post the code since it's under production use case.

I do use a custom client which I register with each agent. I was looking at the issues which request for Ollama support for autogen function calling https://github.com/microsoft/autogen/issues/1738, https://github.com/microsoft/autogen/issues/1667, https://github.com/microsoft/autogen/issues/1738 and https://github.com/microsoft/autogen/issues/152

marklysze commented 2 weeks ago

Hey @remidiy, okay, all those issues are quite old and before Ollama was a client class in AutoGen. If you would like help and look at it more closely, feel free to raise an issue on Discord under the issues-and-help channel.

The Ollama client class ("api_type": 'ollama') should work for function calling. Depending on the capability of the model and the use-case, try with and without native tool calling. This is a parameter on the llm_config: "native_tool_calls": True or "native_tool_calls": False

remidiy commented 2 weeks ago

Thanks for the help @marklysze I'm able to make it work with the changes as suggested. The proxy server that I use in prod is similar to the Litellm proxy with additional headers and authentication so the llm_config did the trick.

microsoft / autogen

[Question][Issue]: Open Source Models Ignore Tools #3196

Describe the issue

Steps to reproduce

pure python

llama 3.1

code

output

gemma 2

code

output

Screenshots and logs

lm studio config

llama3

gemma 2 config

Additional Information

Model results

summary

shared code

Llama3.1

output

Llama3:instruct

gemma 2