[Test Case Failure]: FAILED tests/llms/ShuttleAIModel_test.py::

[cobycloud]

Test Case

tests/llms/ShuttleAIModel_test.py:: MULTIPLE RELATED FAILURES

Root Cause

FAILED tests/llms/ShuttleAIModel_test.py::test_no_system_context - KeyError: 'choices' FAILED tests/llms/ShuttleAIModel_test.py::test_nonpreamble_system_context - KeyError: 'choices' FAILED tests/llms/ShuttleAIModel_test.py::test_preamble_system_context - KeyError: 'choices' FAILED tests/llms/ShuttleAIModel_test.py::test_multiple_system_contexts - KeyError: 'choices' FAILED tests/llms/ShuttleAIToolModel_test.py::test_agent_exec - KeyError: 'choices'

Potential Solutions

correct the following: https://github.com/swarmauri/swarmauri-sdk/blob/8b13c9de1e400916f37eb71a5aa54cc6852cf29d/swarmauri/standard/llms/concrete/ShuttleAIModel.py#L74

Additional Context (optional)

No response

Type of Test Case

Unit

[RafaelJohn9]

The KeyError choices is being caused by unsuccessful requests

Main problem

Unsuccessful request happens due to image attribute being null

On logging this is the error you will receive

INFO     root:ShuttleAIModel.py:121
 {'status': 400,
           'error': {'message': 'We could not parse the JSON body of your request.', 
           'type': 'invalid_request_error', 'code': None, 'param': None, 'hint': "This likely means you aren't using your HTTP library correctly. The ShuttleAI API expects a JSON payload, but what was sent was not valid JSON. If you have trouble figuring out how to fix this, please contact us at https://discord.gg/shuttleai"}}

Additionally

Also the raw parameter is problematic same case with the image parameter

Quick Fix

Currently the only way to make this work will be to comment the image attribute inside the payload.

Real Fix

To be discussed

[RafaelJohn9]

Other problems to handle

Also the same KeyError is caused when you don't have enough Tokens in you account.

Logging message

INFO     root:ShuttleAIModel.py:121 
{'status': 402, 
'error': {
                'message': "You don't have enough credits to access this resource. You need at least 3.5e-06 credits, but you only have 0.0.", 
                 'type': 'InsufficientCredits', 'code': None, 'param': None, 'hint': 'You can purchase more credits at https://shuttleai.app/keys.'}}

[cobycloud]

we will continue to let this error bubble as-is at this time.

[cobycloud]

removed L74

here is the new error:

self = ShuttleAIModel(name='shuttle-2-turbo', id='499d6f66-57ff-4e92-8979-7291124565c3', members=[], owner=None, host=None, r...pro-latest', 'gemini-1.0-pro-vision', 'lzlv-70b', 'figgs-rp', 'cinematika-7b'], api_key='***')
conversation = Conversation(name=None, id='e76368c8-6039-4ab7-b959-3617a35e77ed', members=[], owner=None, host=None, resource='Conversation', version='0.1.0', type='Conversation')
temperature = 0.7, max_tokens = 256, top_p = 1, internet = True
citations = True, tone = 'precise', raw = False, image = None
    def predict(self,
        conversation,
        temperature=0.7,
        max_tokens=256,
        top_p=1,
        internet=True,
        citations=True,
        tone='precise',
        raw=False,
        image=None):

        formatted_messages = self._format_messages(conversation.history)

        url = "https://api.shuttleai.app/v1/chat/completions"
        payload = {
            "model": self.name,
            "messages": formatted_messages,
            "max_tokens": max_tokens,
            "temperature": temperature,
            "top_p": top_p,
            "internet": internet,
            "raw": raw,
            "image": image
        }

        if self.name in ['gpt-4-bing', 'gpt-4-turbo-bing']:
            payload['tone'] = tone
            payload['citations'] = citations

        headers = {
            "Authorization": f"***",
            "Content-Type": "application/json",
        }

        response = requests.request("POST", url, json=payload, headers=headers)
        message_content = response.json()
>       conversation.add_message(AgentMessage(content=message_content))
E       pydantic_core._pydantic_core.ValidationError: 1 validation error for AgentMessage
E       content
E         Input should be a valid string [type=string_type, input_value={'status': 400, 'error': .../discord.gg/shuttleai"}}, input_type=dict]
E           For further information visit https://errors.pydantic.dev/2.8/v/string_type
/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/swarmauri/standard/llms/concrete/ShuttleAIModel.py:75: ValidationError

[cobycloud]

The KeyError choices is being caused by unsuccessful requests

Main problem

Unsuccessful request happens due to image attribute being null

On logging this is the error you will receive

INFO     root:ShuttleAIModel.py:121
 {'status': 400,
           'error': {'message': 'We could not parse the JSON body of your request.', 
           'type': 'invalid_request_error', 'code': None, 'param': None, 'hint': "This likely means you aren't using your HTTP library correctly. The ShuttleAI API expects a JSON payload, but what was sent was not valid JSON. If you have trouble figuring out how to fix this, please contact us at https://discord.gg/shuttleai"}}

Additionally

Also the raw parameter is problematic same case with the image parameter

Quick Fix

Currently the only way to make this work will be to comment the image attribute inside the payload.

Real Fix

To be discussed

this is related to ShuttleAIToolModel