ai-forever MERA issues - Githubissues

ai-forever / MERA

MERA (Multimodal Evaluation for Russian-language Architectures) is a new open benchmark for the Russian language for evaluating fundamental models.

MIT License

58 stars 8 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

Update README

#28 LSinev closed 2 months ago
0
Citation update following ACL 2024

#27 LSinev closed 3 months ago
0
This pull request fixes issues with submission format unmatching in 0.3.0 and 0.4.2

#26 thehir0 opened 4 months ago
1
Update USE description

#25 anastasia-kozlova closed 4 months ago
0
Update submission packing script

#24 LSinev closed 5 months ago
0
no targets in rummlu and others benchmarks

#23 thehir0 opened 5 months ago
12
Few fixes

#22 LSinev closed 5 months ago
0
Dataset card update

#21 LSinev closed 5 months ago
0
влияние промпта на результаты бенчмарков

#20 vlsav closed 6 months ago
0
Большие модели, не влезающие в одну карту, не параллелятся на несколько

#19 preduct0r closed 6 months ago
2
Bug fix with wrong input_ids

#18 thehir0 closed 6 months ago
1
Не авторизоваться на сайте mera.a-ai.ru

#17 preduct0r closed 7 months ago
0
Скоринг GGUF моделей

#16 konductor000 closed 7 months ago
1
0.4.0 lm-evaluation-harness

#15 germanjke opened 8 months ago
9
Значения логов бенчмарка

#14 thehumit opened 9 months ago
1
Docs correction

#13 LSinev closed 9 months ago
0
Docs correction

#12 Colindonolwe closed 9 months ago
0
Как бенчмарк закрытой модели, у которой нету метода loglikelihood?

#11 chorus12 opened 9 months ago
1
[Feature Request] Support for OpenAI ChatCompletion models

#10 kristaller486 opened 10 months ago
2
tokenizer does not have a padding token

#9 razikov closed 10 months ago
1
Updates for v1.1.0

#8 LSinev closed 10 months ago
0
Ошибка при сабмитах на mera.a-ai.ru

#7 GorbetskiyDmitriy closed 10 months ago
2
example mmlu gpt-3.5-turbo-1106

#6 mizinovmv closed 9 months ago
0
Как проскорить модель без метода loglikelihood?

#5 GorbetskiyDmitriy closed 10 months ago
1
Как добавить форматирование промпта?

#4 dmitrymailk opened 1 year ago
2
empty value rummlu

#3 mizinovmv closed 9 months ago
3
Fix create submission

#2 meduzick closed 1 year ago
0
fix typo

#1 IvanSedykh closed 11 months ago
0