-
### Describe the feature
I tried the vLLM and LMDeploy using the following command:
```
python run.py \
--datasets humaneval_gen \
--hf-type chat \
--hf-path meta-llama/Meta-Llama-3-…
-
### Question
Hi, thanks for the great work! I have been trying to evaluate llava image captioning on Flickr30k, but I am not able to reproduce the results. While the original llava paper does not hav…
-
For CoQA, in coqa/utils.py, only the last answer of each text (i.e. the answer for the last turn_id, with all the previous questions and answers in the context window) is predicted. On the website of …
-
[x] I checked the [documentation](https://docs.ragas.io/) and related resources and couldn't find an answer to my question.
**Your Question**
what is unclear to you? What would you like to know?
…
-
### Question Validation
- [X] I have searched both the documentation and discord for an answer.
### Question
I designed a chatbot with an Agent to perform a series of actions.
My agent works like…
-
- [x] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug.
**Your Question**
I would like to use Answer Relevance for RAG evaluation in Jap…
-
# URL
- https://arxiv.org/abs/2401.07103
# Affiliations
- Zhen Li, N/A
- Xiaohan Xu, N/A
- Tao Shen, N/A
- Can Xu, N/A
- Jia-Chen Gu, N/A
- Chongyang Tao, N/A
# Abstract
- In the rapidly…
-
* https://en.wikipedia.org/wiki/MLOps
-
[ ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug.
**Describe the bug**
I am trying to use a local LLM in the evaluate function., whe…
-
Please note our paper on evaluation, which could be an important building block for multilingual evaluation and cultural understanding.
[SeaEval for Multilingual Foundation Models: From Cross-Lingu…