I've structured my query_engine to output as a pydantic object:
class TechOffer(BaseModel):
"""Title of technical offer that is relevant to the query including reasons why technical offer is relevant to the query."""
title: str = Field(
description="Title of the technical offer that is relevant to the query."
)
reason: str = Field(
description="Detailed step-by-step reasons why the technical offer are relevant to the query."
)
unique_value_proposition: str = Field(
description="Unique benefits or advantages offered by the technical offer with reference to the query."
)
class TechOfferTitles(BaseModel):
"""List of titles of technical offers that are relevant to the query."""
title_list: List[TechOffer]
When I run eval_result = evaluator.evaluate_response(response=response), I got this error message:
AttributeError: 'TechOfferTitles' object has no attribute 'query_str'
How can I perform faithfulness evaluation when my LLM output is a pydantic object? Thanks!
I've structured my query_engine to output as a pydantic object:
When I run
eval_result = evaluator.evaluate_response(response=response)
, I got this error message:AttributeError: 'TechOfferTitles' object has no attribute 'query_str'
How can I perform faithfulness evaluation when my LLM output is a pydantic object? Thanks!