-
**Describe the bug**
Can't get the automatic language adaption going for testset generation. I retried this about 10 times.
Ragas version: 0.1.4
Python version: 3.11
**Code to Reproduce**
Sha…
-
Hey @acon96, It seems to me that this is not a bug, but an incorrect configuration. I try to run `train.py` script with some params, on my 1 x RTX4090 graphic card. I try to train `llama-3.1-8B` model…
-
**What would you like to be added/modified:**
1. Build a collaborative code intelligent agent alignment dataset for LLMs:
- The dataset should include behavioral trajectories, feedback, and i…
-
**Description:**
Hi,
I am currently using the latest version of Ragas (version 0.1.10) and encountering an error when using Ragas with local LLMs from Ollama. Here is my code:
```python
fr…
-
Could you please share the evaluation scripts and prompts that were used to generate the reported results in the paper?
Various parameters are involved in generating outputs, and it is crucial to …
-
**Describe the bug**
I am trying to run the code in [LLM Optimization with DirectML](https://github.com/microsoft/Olive/tree/d068cf963e0ec1c5d4ecf9b87c2a1e118fe2303f/examples/directml/llm). The `requ…
-
You will see the problem in the text below, this is with using gpt-4o and version 0.5 of agent zero, but have similar issues with other models
User message ('e' to leave):
> Write a college level …
-
[ ] I checked the [documentation](https://docs.ragas.io/) and related resources and couldn't find an answer to my question.
**Your Question**
> “WARNING:ragas.llms.output_parser:Failed to parse …
-
Hi, @haesleinhuepf recommended I post this here.
Because you guys are making this evaluation benchmark public, couldn't LLMs use this as training data and therefore overfit to it?
So, it will be h…
-
Hello 👋
First of all thank you for the great work and evaluation results!
I have understood that in many cases you predicted outputs for each question based on the choice that minimizes the loss…