dosubot[bot]
commented
2 months ago To resolve the AssertionError: "AnswerSimilarity must be set" when using the answer_correctness metric in Ragas version 0.1.16 with Python 3.11.9, you need to ensure that the AnswerSimilarity object is properly initialized. This is required when the second weight in the weights list is not zero.
Here is the relevant part of the code that checks for the AnswerSimilarity object:
if self.answer_similarity is None and self.weights[1] != 0:
self.answer_similarity = AnswerSimilarity(
llm=self.llm, embeddings=self.embeddings
)Make sure that the AnswerSimilarity object is set correctly in the init method of the AnswerCorrectness class. Here is the complete init method for reference:
def init(self, run_config: RunConfig):
super().init(run_config)
if self.answer_similarity is None and self.weights[1] != 0:
self.answer_similarity = AnswerSimilarity(
llm=self.llm, embeddings=self.embeddings
)Ensure that you call the init method with the appropriate RunConfig object before using the answer_correctness metric. This will initialize the AnswerSimilarity object if it is not already set and the second weight is not zero.
Here is an example of how you might initialize and use the AnswerCorrectness metric:
from ragas.run_config import RunConfig
from ragas.metrics._answer_correctness import AnswerCorrectness
# Create a RunConfig object with necessary parameters
run_config = RunConfig(llm=your_llm, embeddings=your_embeddings)
# Initialize the AnswerCorrectness metric
answer_correctness = AnswerCorrectness()
answer_correctness.init(run_config)
# Now you can use the answer_correctness metric without encountering the AssertionErrorReplace your_llm and your_embeddings with the appropriate objects required for your use case [1].
To continue talking to Dosu, mention @dosu.
xidian237
[ ] I have checked the documentation and related resources and couldn't resolve my bug.
Hi experts,
I use the answer_correctness as the metric, but it failed due to AssertionError: "AssertionError: AnswerSimilarity must be set".
Ragas version: 0.1.16 Python version: 3.11.9
Code to Reproduce I want to call the answer_correctness, here is the test code, and, this aserror occurs occasionally.:
Error trace
Additional context I noticed that the AnswerSimilarity will be inited: