-
Next steps for everyone, general task should be to generate queries using fragments of documents and then create ground truth with them or if domain expert (we know) is available to help us, to genera…
-
- [ ] [The Bitter Lesson](http://www.incompleteideas.net/IncIdeas/BitterLesson.html)
# The Bitter Lesson
**DESCRIPTION:**
"The Bitter Lesson
Rich Sutton
March 13, 2019
The biggest lesson that …
-
### How are you running AnythingLLM?
Local development
### What happened?
When the chat mode is set to Query mode, responses are generated even if the content of the question is completely unrelat…
-
I would like to evaluate rag systems with models from Amazon Bedrock. Got an error when running `npx promptfoo@latest eval` with `providers: [bedrock:anthropic.claude-v2]` using the examples in exampl…
-
**Bug Description**
TruLens recorder works only with OpenAI() models. When I try Anthropic or TogetherAI APIs exceptions emerges.
**To Reproduce**
```py
import logging
from time import sleep
…
aabor updated
5 months ago
-
```
def get_trulens_bedrock():
import os
from trulens_eval import Bedrock
import boto3
region =
profile =
os.environ["AWS_PROFILE"] = profile
os.envir…
-
Hi, can I ask a question regarding the evaluating kNN-LM and RetoMaton? I used the preprocessed Wikitext-103 datastores and FAISS index from gpt-2 and distilgpt-2(downloading form the link) and encoun…
-
**Describe the bug**
I'm following the documentation for creating a synthetic dataset [here](https://docs.ragas.io/en/latest/getstarted/testset_generation.html). The generated dataset contains the fo…
-
### Bug Description
Hello,
I am encountering an issue while attempting to run the RAG evaluation metrics, specifically the CorrectnessEvaluator.
- I created an index for the PaulGrahamEssayData…
-
Running through an evaluation error