issues
search
OpenGPTX
/
lm-evaluation-harness
A framework for few-shot evaluation of autoregressive language models.
MIT License
8
stars
8
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Implemented Polyglotoxicityprompts
#109
jjbuschhoff
opened
2 weeks ago
0
Crowspairsde
#108
NAM00
opened
6 months ago
0
task crowspairs de
#107
NAM00
closed
6 months ago
0
`MegatronLMClient` expects cache usage
#106
janEbert
opened
7 months ago
0
Implemented EU22/EU5 translations
#105
jjbuschhoff
opened
8 months ago
0
Add MT-Bench-X to lm-evaluation-harness
#104
lllAlexanderlll
opened
9 months ago
0
Inspect LAMBADA OpenAI fpr HF converted checkpoints
#103
lllAlexanderlll
opened
9 months ago
0
Disable unit tests
#102
malteos
closed
9 months ago
0
Implemented Belebele benchmark
#101
jjbuschhoff
opened
9 months ago
0
Fix unnatural tokenizations if possible
#100
KlaudiaTH
opened
10 months ago
0
New MLMM tasks
#99
malteos
closed
9 months ago
2
Megatronlm client fixes
#98
KlaudiaTH
closed
10 months ago
0
Integrated various German tasks
#97
jjbuschhoff
closed
9 months ago
2
implemented flores200
#96
jjbuschhoff
closed
9 months ago
1
FIX ogx_xquad el,th,tr,ru,ro
#95
thnkinbtfly
closed
10 months ago
0
Megatronlm client
#94
KlaudiaTH
opened
11 months ago
1
Fix caching
#93
janEbert
closed
10 months ago
0
Batch evaluation script
#92
jjbuschhoff
opened
12 months ago
2
preliminary batch script
#91
jjbuschhoff
closed
1 year ago
0
Add Batching to Megatron-LM server
#90
KlaudiaTH
opened
1 year ago
0
Megatronlm client
#89
KlaudiaTH
closed
1 year ago
0
Small fixes for tasks
#88
katrinklug
closed
1 year ago
0
trust_remote_code is not passed to AutoTokenizer.from_pretrained
#87
KlaudiaTH
opened
1 year ago
0
Restructure opengpt-x tasks
#86
katrinklug
closed
1 year ago
3
Add new evaluation tasks and fix README
#85
katrinklug
closed
1 year ago
1
Create output path directory if necessary
#84
janEbert
closed
1 year ago
0
Fix updating task registry too late
#83
janEbert
closed
1 year ago
0
Add perplexity task on arbitrary JSON data
#82
janEbert
closed
12 months ago
1
Upstreaming contributions?
#81
StellaAthena
opened
1 year ago
1
Fix automated checks
#80
rrutmann
closed
1 year ago
0
add --write_detailed_eval_info to dump JSON with completions and deti…
#79
ghstgs
closed
1 year ago
0
germeval2017, germeval2018_coarse - Not working in Offline Mode
#78
KlaudiaTH
closed
10 months ago
1
add multilingual tasks
#77
ghstgs
closed
1 year ago
0
pawsx_de, pawsx_en, x_stance, gnad10 error message for metrics
#76
katrinklug
opened
1 year ago
1
xquad, mlqa and mlsum tasks are not correctly implemented
#75
KlaudiaTH
opened
1 year ago
3
Add another caching mechanism
#74
janEbert
closed
1 year ago
3
Rebase EleutherAI master on top
#73
janEbert
closed
1 year ago
0
Update xnli.py
#72
ghstgs
closed
1 year ago
0
Add evaluations for anli_de
#71
katrinklug
opened
1 year ago
0
Evaluation for german wino_x
#70
katrinklug
closed
1 year ago
0
Enables self-hosted test runner
#69
malteos
opened
1 year ago
0
Add evaluation task wino-x
#68
katrinklug
closed
1 year ago
0
Add evaluations for miam
#67
katrinklug
opened
1 year ago
0
Add evaluations for xcsr
#66
katrinklug
closed
1 year ago
1
Add evaluations for germandpr
#65
katrinklug
opened
1 year ago
0
Add evaluation for fairlex
#64
katrinklug
opened
1 year ago
0
Add evaluations for lextreme
#63
katrinklug
opened
1 year ago
0
Add evaluation tasks with omp dataset
#62
katrinklug
opened
1 year ago
0
Add evaluation for winogrande_X
#61
katrinklug
closed
9 months ago
0
Add evaluation for QG from xglue benchmark
#60
katrinklug
opened
1 year ago
0
Next