explodinggradients / ragas

Supercharge Your LLM Application Evaluations 🚀
https://docs.ragas.io
Apache License 2.0
7.08k stars 719 forks source link

Answer Correctness not working after pydantic update #1441

Open hubkrieb opened 3 weeks ago

hubkrieb commented 3 weeks ago

[ ] I have checked the documentation and related resources and couldn't resolve my bug.

Describe the bug A clear and concise description of what the bug is.

Ragas version: 0.2.0 Python version: 3.12

Error trace AttributeError('StatementsAnswers' object has no attribute 'dicts')

Additional context The dicts method was deleted from the StatementAnswers class after the pydantic update 95f8d7bf590f19f1d0fc302d3e64556ae5e49f18

dosubot[bot] commented 3 weeks ago

@hubkrieb Hello there! I'm here to help you with any bugs, questions, or contributions you might have. Let's tackle this together!

To resolve the AttributeError caused by the removal of the dicts method from the StatementsAnswers class after the pydantic update in Ragas version 0.2.0 with Python 3.12, you should replace the dicts method with the model_dump method. The model_dump method is used to serialize the model to a dictionary, which is the intended functionality of the removed dicts method.

Here's how you can modify the code:

Replace any instance of answers.dicts() with answers.model_dump().

For example, in the _compute_score method, change:

faithful_statements = sum(
    1 if answer.verdict else 0 for answer in answers.model_dump()
)
num_statements = len(answers.model_dump())

Additionally, ensure that any other usage of dicts() in the code is replaced with model_dump(), as seen in the NoiseSensitivity class methods like _evaluate_statement_faithfulness and _decompose_answer_into_statements [1][2].

To continue talking to Dosu, mention @dosu.