[BUG]Groundedness / ground_truth_agreement not working

[hackrz]

Bug Description Unable to run groundedness / ground_truth_agreement without a glitch

To Reproduce ground_truth_agreement = ( Feedback(GEMINI_PROVIDER.groundedness_measure_with_cot_reasons, name="Groundedness") .on(TruLlama.select_source_nodes().node.text) .on_output() .aggregate(GroundTruthAgreement.boolean_and_aggregator) )

Expected behavior The code should run in Pycharm community edition 2024.1 without an error. The code is import os import google.generativeai as genai import nest_asyncio import numpy as np from trulens_eval import Feedback, LiteLLM, TruLlama

from trulens_eval.feedback import Groundedness

from trulens_eval.feedback import GroundTruthAgreement

genai.configure(api_key=os.environ['GOOGLE_API_KEY']) nest_asyncio.apply() GEMINI_PROVIDER = LiteLLM(model_engine='gemini-1.5-flash')

ground_truth_agreement = ( Feedback(GEMINI_PROVIDER.groundedness_measure_with_cot_reasons, name="Groundedness") .on(TruLlama.select_source_nodes().node.text) .on_output() .aggregate(GroundTruthAgreement.boolean_and_aggregator) )

Relevant Logs/Tracebacks .aggregate(GroundTruthAgreement.boolean_and_aggregator) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/abc/PycharmProjects/rag_training/.venv/lib/python3.12/site-packages/pydantic/_internal/_model_construction.py", line 242, in getattr raise AttributeError(item) AttributeError: boolean_and_aggregator

Environment:

• OS: MacOS
• Python Version: 3.12
• Langchain: 0.2.6
• Langchain- community: 0.2.6
• langchain cli: 0.0.25
• trulens_eval: 0.32.0

Additional context I also tried following from DLAI training and github closed issues but no luck: Option 1: ground_truth_agreement = ( Feedback(GEMINI_PROVIDER.ground_truth_measure_with_cot_reasons, name="Ground Truth Agreement") .on(TruLlama.select_source_nodes().node.text) .on_output() .aggregate(GroundTruthAgreement.boolean_and_aggregator) )

Option 2: f_groundedness = ( Feedback(provider.groundedness_measure_with_cot_reasons, name = "Groundedness") .on(Select.RecordCalls.retrieve.rets.collect()) .on_output() ) Option 1 does not recognise. "ground_truth_measure_with_cot_reasons" and option 2 does not recognise "Select"

[dosubot[bot]]

The AttributeError: boolean_and_aggregator occurs because the GroundTruthAgreement class does not have a method or attribute named boolean_and_aggregator. To resolve this issue, you need to use a valid aggregation method available in the GroundTruthAgreement class.

Here are the available aggregation methods in the GroundTruthAgreement class:

1. agreement_measure
2. mae
3. bert_score
4. bleu
5. rouge

You can use one of these methods for aggregation. For example, if you want to use the agreement_measure method, you can modify your code as follows:

import os
import google.generativeai as genai
import nest_asyncio
import numpy as np
from trulens_eval import Feedback, LiteLLM, TruLlama
from trulens_eval.feedback import GroundTruthAgreement

genai.configure(api_key=os.environ['GOOGLE_API_KEY'])
nest_asyncio.apply()
GEMINI_PROVIDER = LiteLLM(model_engine='gemini-1.5-flash')

ground_truth_agreement = (
    Feedback(GEMINI_PROVIDER.groundedness_measure_with_cot_reasons, name="Groundedness")
    .on(TruLlama.select_source_nodes().node.text)
    .on_output()
    .aggregate(GroundTruthAgreement.agreement_measure)
)

This should resolve the AttributeError and allow your code to run without errors ^[1].

To continue talking to Dosu, mention @dosu.