-
[ ] I checked the [documentation](https://docs.ragas.io/) and related resources and couldn't find an answer to my question.
**Your Question**
what is unclear to you? What would you like to know?
…
-
[ ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug.
**Describe the bug**
I am trying to generate test data for ragas evaluation, I hav…
-
### Question Validation
- [X] I have searched both the documentation and discord for an answer.
### Question
When evaluating a RAG retrieval service using the llama-index evaluation method, I encou…
-
Thank you for your excellent work!
In the current implementation of `qa_evaluate`, the `has_intersection` function used for comparing predicted answers with gold standard answers splits strings on …
-
This is a thread for Carolina to summarize her research on RAG.
The purpose is to share the information among project members.
-
### Bug Description
Following the tutorial on LATS agent (https://docs.llamaindex.ai/en/stable/examples/agent/lats_agent/) result in the RuntimeError : Event loop is closed
### Version
0.11.12
##…
-
Hello 👋
First of all thank you for the great work and evaluation results!
I have understood that in many cases you predicted outputs for each question based on the choice that minimizes the loss…
-
Lastly i was looking at the `YAML` files for the **UnifyAI** tools, and I had a few ideas that might help:
**Here are my thoughts 🤔**:
- `evaluate_llm_tool.yaml:`
It seems like the `prompt…
-
G-Eval includes "Auto Chain-of-Thoughts for NLG Evaluation" as a component where the CoT steps to carry out evaluation are produced by an LLM. The paper nor this repo, however, include the prompt defi…
-
Estimate key LLM metrics:
- Overall quality score, accuracy
- Hallucination rate (hallucination detection)
- Relevancy
- Coherence
- Responsible AI violations
- Safety