-
Hello, i created a testset and run it through my RAG pipeline to get documents and a answer for each question. I now have 50 pairs of [question, ground_truth, documents, answer] that i want get the co…
-
Step 1: Retrieval
- Measure ranking metrics
- Success rate / hit-rate
- MRR
Step 2: Synthesis
-
**Describe the bug**
Sometimes, the evaluator returns null for score/explanation and sometimes it crashes with the following error:
```2024-03-10 22:36:45.719 | INFO | uptrain.framework.evalll…
lbux updated
5 months ago
-
Fine-tuning:
- 4 models are working on Ollama (3 tinyLlama verisons with 1, 10, 50 epoch)
- I was able to train an Llama2 model (1 epoch only)
- Llama.cpp depricated some functionality which made …
-
Hi,
Thanks for your contribution, it's really useful to see evaluations on real-world data! There are further extraction tools for Python which this repository doesn't feature yet and which could b…
adbar updated
4 months ago
-
Hi, I'm working with the LLMTestCase example:
```python
test_case = LLMTestCase(
input="What if these shoes don't fit?",
# Substitute this with the actual output from your LLM tool
…
-
Hi,
I am using partition and chunk_by_title to chunk my pdfs. It generally works but when I investigated the chunks I saw that if there is a Table in one of my documents, the title of the table is …
-
Hello everyone,
I've developed two RAG (Naive RAG and Advance RAG) applications using the following technologies:
**LLM Framework**: llama-index
**LLM Model**: Llama3.1 (8b) using Ollama
**Embe…
-
Setup:
- Windows Desktop App 1.6.4
- Open new work space, set the default settings with local Qwen2 7b instruct fp16 model for Chat and Agents
- -Chat Mode: Chat
- Use Paperclip icon or drag and d…
-
[ yes] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug.
**Describe the bug**
I keep getting missing required fields when trying to use …