-
### Bug Description
Hello,
I am encountering an issue while attempting to run the RAG evaluation metrics, specifically the CorrectnessEvaluator.
- I created an index for the PaulGrahamEssayData…
-
```
def get_trulens_bedrock():
import os
from trulens_eval import Bedrock
import boto3
region =
profile =
os.environ["AWS_PROFILE"] = profile
os.envir…
-
**Bug Description**
TruLens recorder works only with OpenAI() models. When I try Anthropic or TogetherAI APIs exceptions emerges.
**To Reproduce**
```py
import logging
from time import sleep
…
aabor updated
6 months ago
-
Hi, can I ask a question regarding the evaluating kNN-LM and RetoMaton? I used the preprocessed Wikitext-103 datastores and FAISS index from gpt-2 and distilgpt-2(downloading form the link) and encoun…
-
Running through an evaluation error
-
I'm currently working on LLM evaluation using Trulens and Azure openai and encountering some issues with the code. Here's the relevant portion of my code:
```
retriever = vectorstore .as_retrieve…
-
### Issue you'd like to raise.
When i try to customize the LLM running the evaluation, i get the test to run without failling but it did not save the scores in Langsmith like it normaly does when i r…
-
After the updates from https://github.com/Azure-Samples/ai-rag-chat-evaluator/pull/45 I've rerun the evaluation and got the minor score for all metrics:
Before update:
(I've omitted the context)…
-
Hi:
Really interesting work on RAG evaluation metrics. However, I am unable to understand how these evaluation functions work. Could you please point me to some relevant documentation that explains…
vr25 updated
10 months ago
-
Hello,
Is there a way to track progress when giving a list of inputs to a LLMChain object using tqdm for example?
I didn't see any parameter that would allow me to use tqdm.
I also checked if I c…