-
## Adding a Dataset
- **Name:** *multilingual_mmlu_th*
- **Dataset Description:** *Thai mmlu from META*
- **Dataset URL:** *[original URL of the dataset](https://huggingface.co/datasets/meta-llama/…
-
With [HIT-TMG/KaLM-embedding-multilingual-mini-instruct-v1](https://huggingface.co/HIT-TMG/KaLM-embedding-multilingual-mini-instruct-v1), I have the following error:
```
Loader not specified for m…
-
- Shouldn't this be Touche2020: https://github.com/embeddings-benchmark/mteb/blob/a988fef10cb73e2a35238f14f5c59a6615bbdaeb/mteb/benchmarks/benchmarks.py#L189 not the new one like here https://github.c…
-
Please note our paper on evaluation, which could be an important building block for multilingual evaluation and cultural understanding.
[SeaEval for Multilingual Foundation Models: From Cross-Lingu…
-
Hello, the data set of livecodebench is Python, would you consider supporting multi-language data set evaluation? Especially Java. thanks.
-
Issue is to track evaluation of RAG implementations.
Frameworks:
- RAGEval
- https://github.com/OpenBMB/RAGEval
- https://arxiv.org/pdf/2408.01262
- AutoRAG
- https://github.com/Marker-Inc-K…
-
## Evaluation short description
- Why is this evaluation interesting?
This focuses on 16 African languages, evaluated on three knowledge QA and reasoning tasks such as AfriMMLU, AfriMGSM and AfriXNL…
-
# Task Name
Multilingual Speech to Speech Translation (s2st): converting speech from one language directly into speech in another language. This task requires the model to have strong multilingual …
-
**Background**
In the field of multilingual large models, especially for non-English corpora, there is often a problem of insufficient data quantity and poor quality. High-quality training data is cr…
-
We have a new project involving multilingual retrieval and reproduction and we are looking for 2 URA students to work together.
Feel free to reach out on Slack or email us at nandant@gmail.com, xzh…