-
## Description
One of the difficulties with using LLM-as-judge evaluations is trusting their determinations. Therefore, it's important to select an evaluator LLM (different from the LLMs that will be …
-
-
### Bug Description
So I want to get the metrics from RAGAS and whenever I dowwnload model locally in my EC2 the metrics gives me the answer biut when I use sagemaker end point as a LLM it thros er…
-
[ ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug.
**Describe the bug**
```
RuntimeError: There is no current event loop in thread '…
-
# Evaluating the Effectiveness of LLM-Evaluators (aka LLM-as-Judge)
Use cases, techniques, alignment, finetuning, and critiques against LLM-evaluators.
[https://eugeneyan.com/writing/llm-evaluators/…
-
**Describe the bug**
I am trying to run the code in [LLM Optimization with DirectML](https://github.com/microsoft/Olive/tree/d068cf963e0ec1c5d4ecf9b87c2a1e118fe2303f/examples/directml/llm). The `requ…
-
I want to use a specific ```model:tag``` provided by ollama.
Ref
- https://ollama.com/library/qwen2:72b
However, if i use the ollama specific tag as shown below, i got an error.
```
mod…
-
I am interested in using Uptrain with a locally hosted open-source LLM as evaluator LLM. I'm currently hosting a LLM service using vLLM, not with Ollama. Is there any way to use this local LLM with U…
-
**Describe the bug**
A clear and concise description of what the bug is.
I tried using RAGAS with a model that is not OpenAI. In general whatever model I use I get this error back:
```
File …
-
Hi! I am wondering if it's possible to use open source or self-deployed llms (and not only open ai) as the judge or evaluator? If yes, could you please refer to an example or part of the docs explaini…