APIError: litellm.APIError: APIError: GroqException - You tried to pass a `BaseModel` class to `chat.completions.create()`; You must use `beta.chat.completions.parse()` instead

[plaban1981]

Error Encountered :

APIError: litellm.APIError: APIError: GroqException - You tried to pass a BaseModel class to chat.completions.create(); You must use beta.chat.completions.parse() instead during detecting Hallucination using opik libaray

06:41:19 - LiteLLM:WARNING: utils.py:290 - `litellm.set_verbose` is deprecated. Please set `os.environ['LITELLM_LOG'] = 'DEBUG'` for debug logs.
WARNING:LiteLLM:`litellm.set_verbose` is deprecated. Please set `os.environ['LITELLM_LOG'] = 'DEBUG'` for debug logs.

Request to litellm:
litellm.completion(model='groq/llama-3.1-70b-versatile', messages=[{'content': 'You are an expert judge tasked with evaluating the faithfulness of an AI-generated answer to the given context. Analyze the provided INPUT, CONTEXT, and OUTPUT to determine if the OUTPUT contains any hallucinations or unfaithful information.\n\nGuidelines:\n1. The OUTPUT must not introduce new information beyond what\'s provided in the CONTEXT.\n2. The OUTPUT must not contradict any information given in the CONTEXT.\n2. The OUTPUT should not contradict well-established facts or general knowledge.\n3. Ignore the INPUT when evaluating faithfulness; it\'s provided for context only.\n4. Consider partial hallucinations where some information is correct but other parts are not.\n5. Pay close attention to the subject of statements. Ensure that attributes, actions, or dates are correctly associated with the right entities (e.g., a person vs. a TV show they star in).\n6. Be vigilant for subtle misattributions or conflations of information, even if the date or other details are correct.\n7. Check that the OUTPUT doesn\'t oversimplify or generalize information in a way that changes its meaning or accuracy.\n\nAnalyze the text thoroughly and assign a hallucination score between 0 and 1, where:\n- 0.0: The OUTPUT is entirely faithful to the CONTEXT\n- 1.0: The OUTPUT is entirely unfaithful to the CONTEXT\n\n\n\nINPUT (for context only, not to be used for faithfulness evaluation):\nWhat is the capital of France?\n\nCONTEXT:\n[\'France is a country in Western Europe. Its capital is Paris, which is known for landmarks like the Eiffel Tower.\']\n\nOUTPUT:\nThe capital of France is Paris. It is famous for its iconic Eiffel Tower and rich cultural heritage.\n\nProvide your verdict in JSON format:\n{\n    "score": <your score between 0.0 and 1.0>,\n    "reason": [\n        <list your reasoning as bullet points>\n    ]\n}', 'role': 'user'}], response_format=<class 'opik.evaluation.metrics.llm_judges.hallucination.metric.HallucinationResponseFormat'>)

SYNC kwargs[caching]: False; litellm.cache: None; kwargs.get('cache')['no-cache']: False
Final returned optional params: {'response_format': <class 'opik.evaluation.metrics.llm_judges.hallucination.metric.HallucinationResponseFormat'>, 'extra_body': {}}

POST Request Sent from LiteLLM:
curl -X POST \
https://api.groq.com/openai/v1/ \
-d '{'model': 'llama-3.1-70b-versatile', 'messages': [{'content': 'You are an expert judge tasked with evaluating the faithfulness of an AI-generated answer to the given context. Analyze the provided INPUT, CONTEXT, and OUTPUT to determine if the OUTPUT contains any hallucinations or unfaithful information.\n\nGuidelines:\n1. The OUTPUT must not introduce new information beyond what\'s provided in the CONTEXT.\n2. The OUTPUT must not contradict any information given in the CONTEXT.\n2. The OUTPUT should not contradict well-established facts or general knowledge.\n3. Ignore the INPUT when evaluating faithfulness; it\'s provided for context only.\n4. Consider partial hallucinations where some information is correct but other parts are not.\n5. Pay close attention to the subject of statements. Ensure that attributes, actions, or dates are correctly associated with the right entities (e.g., a person vs. a TV show they star in).\n6. Be vigilant for subtle misattributions or conflations of information, even if the date or other details are correct.\n7. Check that the OUTPUT doesn\'t oversimplify or generalize information in a way that changes its meaning or accuracy.\n\nAnalyze the text thoroughly and assign a hallucination score between 0 and 1, where:\n- 0.0: The OUTPUT is entirely faithful to the CONTEXT\n- 1.0: The OUTPUT is entirely unfaithful to the CONTEXT\n\n\n\nINPUT (for context only, not to be used for faithfulness evaluation):\nWhat is the capital of France?\n\nCONTEXT:\n[\'France is a country in Western Europe. Its capital is Paris, which is known for landmarks like the Eiffel Tower.\']\n\nOUTPUT:\nThe capital of France is Paris. It is famous for its iconic Eiffel Tower and rich cultural heritage.\n\nProvide your verdict in JSON format:\n{\n    "score": <your score between 0.0 and 1.0>,\n    "reason": [\n        <list your reasoning as bullet points>\n    ]\n}', 'role': 'user'}], 'response_format': <class 'opik.evaluation.metrics.llm_judges.hallucination.metric.HallucinationResponseFormat'>, 'extra_body': {}}'

openai.py: Received openai error - You tried to pass a `BaseModel` class to `chat.completions.create()`; You must use `beta.chat.completions.parse()` instead

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Provider List: https://docs.litellm.ai/docs/providers

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
[/usr/local/lib/python3.10/dist-packages/litellm/llms/OpenAI/openai.py](https://localhost:8080/#) in completion(self, model_response, timeout, optional_params, logging_obj, model, messages, print_verbose, api_key, api_base, acompletion, litellm_params, logger_fn, headers, custom_prompt_dict, client, organization, custom_llm_provider, drop_params)
    853                     else:
--> 854                         raise e
    855         except OpenAIError as e:

18 frames
TypeError: You tried to pass a `BaseModel` class to `chat.completions.create()`; You must use `beta.chat.completions.parse()` instead

During handling of the above exception, another exception occurred:

OpenAIError                               Traceback (most recent call last)
OpenAIError: You tried to pass a `BaseModel` class to `chat.completions.create()`; You must use `beta.chat.completions.parse()` instead

During handling of the above exception, another exception occurred:

APIError                                  Traceback (most recent call last)
[/usr/local/lib/python3.10/dist-packages/litellm/litellm_core_utils/exception_mapping_utils.py](https://localhost:8080/#) in exception_type(model, original_exception, custom_llm_provider, completion_kwargs, extra_kwargs)
    402                     else:
    403                         exception_mapping_worked = True
--> 404                         raise APIError(
    405                             status_code=original_exception.status_code,
    406                             message=f"APIError: {exception_provider} - {message}",

APIError: litellm.APIError: APIError: GroqException - You tried to pass a `BaseModel` class to `chat.completions.create()`; You must use `beta.chat.completions.parse()` instead

Code Implemented

from opik.evaluation.metrics import Hallucination
import litellm
litellm.set_verbose=True
metric = Hallucination(model='groq/llama-3.1-70b-versatile')

metric.score(
    input="What is the capital of France?",
    output="The capital of France is Paris. It is famous for its iconic Eiffel Tower and rich cultural heritage.",
    context=["France is a country in Western Europe. Its capital is Paris, which is known for landmarks like the Eiffel Tower."],
)

Could you please help me with the right way of using LLMs other than OpenAI Thanks,

[krrishdholakia]

what version of litellm is this? we should already be handling this conversion

[plaban1981]

what version of litellm is this? we should already be handling this conversion

Litellm==1.52.12

[krrishdholakia]

able to repro. Groq doesn't support json schema in the response_format param yet - https://console.groq.com/docs/api-reference#chat-create

we can workaround this with tool calling

[plaban1981]

@krrishdholakia How do I proceed can you please help me with an example