kevintsai / Building-and-Evaluating-Advanced-RAG-Applications

Jupyter notebooks for course Building and Evaluating Advanced RAG Applications, taught by Jerry Liu (Co-founder and CEO of LlamaIndex) and Anupam Datta (Co-founder and chief scientist of TruEra).
https://www.deeplearning.ai/short-courses/building-evaluating-advanced-rag/
38 stars 14 forks source link

utils.py & .env.example not included #2

Open scottpatrickwright opened 8 months ago

scottpatrickwright commented 8 months ago

The utils.py dependency is missing.

FengD commented 3 months ago

The utils.py dependency is missing.

Hi, I followed the course of LlamaIndex and met the same problems. You could follow this link of the trulens. https://www.trulens.org/trulens_eval/getting_started/core_concepts/1_rag_prototype/

I put my code here. And works at my side.

import numpy as np
from trulens_eval import Tru, Feedback, TruLlama, OpenAI as fOpenAI

provider = OpenAI()
context = TruLlama.select_context()

answer_relevance = (
    Feedback(provider.relevance_with_cot_reasons, name="Answer Relevance")
    .on_input_output()
)

context_relevance = (
    Feedback(provider.context_relevance_with_cot_reasons, name = "Context Relevance")
    .on_input()
    .on(context)
    .aggregate(np.mean)
)

f_groundedness = (
    Feedback(provider.groundedness_measure_with_cot_reasons, name="Groundedness")
        .on(context.collect())
        .on_output()
)

def get_prebuilt_trulens_recorder(query_engine, app_id):
    tru_recorder = TruLlama(query_engine,
        app_id='Direct Query Engine',
        feedbacks=[answer_relevance, context_relevance, f_groundedness])

    return tru_recorder

tru_recorder = get_prebuilt_trulens_recorder(query_engine, app_id="Direct Query Engine")