-
[ ] I checked the [documentation](https://docs.ragas.io/) and related resources and couldn't find an answer to my question.
**Your Question**
> “WARNING:ragas.llms.output_parser:Failed to parse …
-
### Prerequisite
- [X] I have searched [Issues](https://github.com/open-compass/opencompass/issues/) and [Discussions](https://github.com/open-compass/opencompass/discussions) but cannot get the expe…
-
As a user, I would like to be informed about the summarization effectiveness of my chosen LLM endpoint.
I would like to be able to evaluate an endpoint against a known, tested framework, to evaluat…
-
When I run `evaluate` with any model of VertexAI, I get several warnings that say
> Gapic client context issue detected.This can occur due to parallelization.
And sometimes the execution of eva…
-
Currently, Unsloth can only support single GPU training, how can you implement it with 8-GPU training? Thx
-
In [continuous_eval/metrics/generation/text/llm_based.py](https://github.com/relari-ai/continuous-eval/blob/main/continuous_eval/metrics/generation/text/llm_based.py), line 39, the full model paramete…
-
### What happened?
The latest llama.cpp produces bad outputs for CodeShell, which previously performed well when merged into llama.cpp.
After updating `convert-hf-to-gguf.py` and `convert-hf-to-g…
-
when i run this cmd on linux:
python -m groma.eval.run_groma \
--model-name /mnt/hgfs/E/1MyFiles/code/aiTest/llm_groma-7b-finetune \
--image-file /home/kun/Downloads/arm_vlm/…
-
-
### 🚀 The feature
Right now eval is done using GPT4 model.
Can we use any open source model to do the same?
### Motivation, pitch
from slack user Bojan Jakimovski