Create a ShuttleAIModel for chatcompletions.

[cobycloud]

This will be for the standard/llms/concrete/ShuttleAIModel

The API KEY name in the test file should be: SHUTTLEAI_API_KEY

Here is documentation: https://docs.shuttleai.app/getting-started/introduction

Our focus will only be on the chat completions endpoint: https://docs.shuttleai.app/api-reference/endpoint/chat-completion

The allowed models should include:

shuttle-2-turbo shuttle-turbo gpt-4o-2024-05-13 gpt-4-turbo-2024-04-09 gpt-4-0125-preview gpt-4-1106-preview gpt-4-1106-vision-preview gpt-4-0613 gpt-4-bing gpt-4-turbo-bing gpt-4-32k-0613 gpt-3.5-turbo-0125 gpt-3.5-turbo-1106 claude-3-opus-20240229 claude-3-sonnet-20240229 claude-3-haiku-20240307 claude-2.1 claude-2.0 claude-instant-1.2 claude-instant-1.1 claude-instant-1.0 meta-llama-3-70b-instruct meta-llama-3-8b-instruct llama-3-sonar-large-32k-online llama-3-sonar-small-32k-online llama-3-sonar-large-32k-chat llama-3-sonar-small-32k-chat blackbox blackbox-code wizardlm-2-8x22b wizardlm-2-70b dolphin-2.6-mixtral-8x7b codestral-latest mistral-large mistral-next mistral-medium mistral-small mistral-tiny mixtral-8x7b-instruct-v0.1 mixtral-8x22b-instruct-v0.1 mistral-7b-instruct-v0.2 mistral-7b-instruct-v0.1 nous-hermes-2-mixtral-8x7b gemini-1.5-pro-latest gemini-1.0-pro-latest gemini-1.0-pro-vision lzlv-70b figgs-rp cinematika-7b

[cobycloud]

The test file will require parameterization and should include all free APIs: shuttle-2-turbo shuttle-turbo gpt-4-0613 gpt-4-bing gpt-4-turbo-bing gpt-3.5-turbo-0125 gpt-3.5-turbo-1106 claude-instant-1.0 wizardlm-2-8x22b wizardlm-2-70b dolphin-2.6-mixtral-8x7b mistral-medium mixtral-8x7b-instruct-v0.1 mixtral-8x22b-instruct-v0.1 mistral-7b-instruct-v0.2 mistral-7b-instruct-v0.1 nous-hermes-2-mixtral-8x7b

[cobycloud]

An example of parameterization:

To execute a pytest while changing one variable each time, the best approach is to use parameterization. Pytest allows you to run a test function multiple times with different sets of arguments using the @pytest.mark.parametrize decorator. This way, you can easily vary one or more variables across different test runs.

Here’s a step-by-step guide on how to do it:

1. Define Your Test Function First, define your test function that accepts the variable(s) you want to change.

def function_to_test(variable):
    # Your function logic here
    return variable * 2  # Example logic

2. Use @pytest.mark.parametrize to Change the Variable Use the @pytest.mark.parametrize decorator to specify the variable and the values you want to test.

import pytest

@pytest.mark.parametrize("variable", [1, 2, 3, 4, 5])
def test_function_to_test(variable):
    result = function_to_test(variable)
    assert result == variable * 2  # Example assertion

@pytest.mark.parametrize("variable", [1, 2, 3, 4, 5]): This line tells pytest to run test_function_to_test 5 times, each time with variable set to one of the values in the list [1, 2, 3, 4, 5].

test_function_to_test(variable): This is your test function that will be run multiple times with the different values of variable.

assert result == variable * 2: This is an example assertion that checks if the function behaves as expected for each value of variable.

Handling Multiple Variables If you need to vary more than one variable, you can pass multiple arguments to @pytest.mark.parametrize. For example:

@pytest.mark.parametrize("variable1, variable2", [(1, 2), (3, 4), (5, 6)])
def test_function_to_test(variable1, variable2):
    result = function_to_test(variable1, variable2)
    assert result == expected_value  # Replace with actual logic

This will run the test function for each pair of (variable1, variable2) values.

[cobycloud]

we will parameterize in a later version