Replace context with retrieval_context in LLMTestCaseParams for HallucinationMetric

[louisbrulenaudet]

Replace context with retrieval_context in LLMTestCaseParams for HallucinationMetric to align with other classes and implement a loop over evaluators using the same TestCase.

Previously, this class was the only one not to borrow this nomenclature, which blocked the possibility of producing loops on the same TestCase. Now, codes like the following are possible

requests = [
    request
] if isinstance(request, EvaluationRequest) else request

for req in tqdm(requests):
    test_case = LLMTestCase(
        input=req.question,
        actual_output=req.output,
        expected_output=req.expected_output or None,
        retrieval_context=req.contexts if req.contexts else None,
    )
    for metric in tqdm(self.metrics):
        instance = self.metrics_config.get_metric_instance(
            metric, self.client
        )

        instance.measure(test_case)

        self.evaluations.append(
            EvaluationResult(
                request=req,
                evaluator_type=metric,
                overall_score=instance.score,
                feedback=instance.reason,
            )
        )

Let me know if you have any questions.

Louis Brulé Naudet

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
evals-docs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Nov 13, 2024 8:41pm

[penguine-ip]

@louisbrulenaudet What is the piece of code you're showing^ ?

Hallucination uses context while faithfulness will use retrieval context: https://docs.confident-ai.com/docs/metrics-faithfulness, not sure if you discovered this.