Closed cobycloud closed 3 weeks ago
tests/llms/ShuttleAIToolModel_test.py::test_ubc_type PASSED [ 58%]
tests/llms/ShuttleAIToolModel_test.py::test_serialization PASSED [ 58%]
tests/llms/ShuttleAIToolModel_test.py::test_default_name PASSED [ 58%]
tests/llms/ShuttleAIToolModel_test.py::test_agent_exec
-------------------------------- live log call ---------------------------------
{'id': 'chatcmpl-cdfef69f08784405aedcb2d636a14aba', 'object': 'chat.completion', 'created': 1724159258, 'model': 'shuttle-2.5', 'choices': [{'index': 0, 'message': {'role': 'assistant', 'content': "I apologize, but it seems there was a mistake in the search query. I was asked to perform an addition of 512 and 671, not to search for any specific information. The search results provided are not relevant to the original request. Please let me know if you need assistance with the math problem or if there's anything else I can help you with."}, 'finish_reason': 'stop'}], 'usage': {'prompt_tokens': 39, 'completion_tokens': 77, 'total_charged': 7.34e-05}}
messages: [{'content': 'Add 512+671', 'role': 'user'}, "I apologize, but it seems there was a mistake in the search query. I was asked to perform an addition of 512 and 671, not to search for any specific information. The search results provided are not relevant to the original request. Please let me know if you need assistance with the math problem or if there's anything else I can help you with."]
agent_response: {'id': 'chatcmpl-cdfef69f08784405aedcb2d636a14aba', 'object': 'chat.completion', 'created': 1724159258, 'model': 'shuttle-2.5', 'choices': [{'index': 0, 'message': {'role': 'assistant', 'content': "I apologize, but it seems there was a mistake in the search query. I was asked to perform an addition of 512 and 671, not to search for any specific information. The search results provided are not relevant to the original request. Please let me know if you need assistance with the math problem or if there's anything else I can help you with."}, 'finish_reason': 'stop'}], 'usage': {'prompt_tokens': 39, 'completion_tokens': 77, 'total_charged': 7.34e-05}}
conversation: name=None id='05f1fff5-4366-45a7-93fb-14a277966218' members=[] owner=None host=None resource='Conversation' version='0.1.0' type='Conversation'
I apologize, but it seems there was a mistake in the search query. I was asked to perform an addition of 512 and 671, not to search for any specific information. The search results provided are not relevant to the original request. Please let me know if you need assistance with the math problem or if there's anything else I can help you with.
PASSED ```
Test Case
tests/llms/ShuttleAIToolModel_test.py::test_agent_exec
Root Cause
Potential Solutions
TBD
Additional Context (optional)
No response
Type of Test Case
Unit