issues
search
GEM-benchmark
/
GEM-metrics
Automatic metrics for GEM tasks
https://gem-benchmark.com
MIT License
60
stars
20
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Unit tests do not consider cache
#54
ndaheim
closed
2 years ago
2
Cannot run meteor evaluation
#53
lxuechen
closed
3 years ago
2
Fixes #51 (Single file mode)
#52
tuetschek
closed
2 years ago
1
Cannot run example command with example data
#51
lxuechen
closed
2 years ago
0
New version of QuestEval
#50
notricenotsweet
closed
3 years ago
0
Add PRISM
#49
tuetschek
closed
2 years ago
0
Speedups
#48
sebastianGehrmann
closed
3 years ago
6
Changing heavy metrics dependencies to an optional feature
#47
tuetschek
closed
3 years ago
0
CLIPScore:A Reference-free Evaluation Metric for Image Captioning
#46
kaustubhdhole
opened
3 years ago
0
Release some starting guide for participants
#45
sebastianGehrmann
opened
3 years ago
1
AttributeError: 'list' object has no attribute 'numpy' in bertscore.py
#44
pasricha
closed
3 years ago
0
Use eval_config to detect languages and tasks automatically.
#43
tuetschek
closed
3 years ago
0
Nubia fails for long texts
#42
tuetschek
closed
3 years ago
6
Add NUBIA
#41
aliabd
closed
3 years ago
0
Questeval
#40
padipadou
closed
3 years ago
9
Installation
#39
tuetschek
closed
3 years ago
0
Add method to allow evaluation on set of examples with IDs
#38
sebastianGehrmann
closed
3 years ago
0
Add unit tests #26
#37
madaan
closed
3 years ago
3
Add NUBIA
#36
sebastianGehrmann
closed
3 years ago
1
Add NIST
#35
sebastianGehrmann
closed
3 years ago
2
Add MoverScore
#34
sebastianGehrmann
closed
2 years ago
2
Adding WER/TER/CIDEr
#33
sebastianGehrmann
closed
2 years ago
4
Pass window_size = 100 as a default parameter to MSTTR.compute
#32
madaan
closed
3 years ago
4
Add check for ngram_len for calculating distinct ngrams
#31
madaan
closed
3 years ago
3
Type hints
#30
madaan
opened
3 years ago
0
Readme for adding metrics
#29
tuetschek
closed
2 years ago
1
Local Recall fails for different numbers of references
#28
tuetschek
closed
3 years ago
2
Czech restaurant (and possibly DART) BLEU seems suspiciously low
#27
tuetschek
opened
3 years ago
14
Adding unit tests
#26
madaan
closed
3 years ago
0
Fail gracefully if Punkt does not support the language
#25
tuetschek
closed
3 years ago
4
Additional diversity metrics
#24
evanmiltenburg
opened
3 years ago
2
Parameter number as descriptive metric
#23
sebastianGehrmann
closed
3 years ago
4
Adding standard deviation etc of prediction lengths
#22
evanmiltenburg
closed
3 years ago
0
Dataset-specific metrics
#21
evanmiltenburg
opened
3 years ago
4
Fix for #18
#20
evanmiltenburg
closed
3 years ago
0
Add verbose mode
#19
evanmiltenburg
opened
3 years ago
0
ValueError in meteor.py
#18
evanmiltenburg
closed
3 years ago
8
Refactored Global Metrics List
#17
nishantsubramani
closed
3 years ago
1
Add local recall metric
#16
evanmiltenburg
closed
3 years ago
5
Add License
#15
evanmiltenburg
closed
3 years ago
2
Add PARENT
#14
sebastianGehrmann
opened
3 years ago
4
Add SARI metric
#13
ddhruvkr
closed
3 years ago
13
Fix running on GPU
#12
tuetschek
closed
3 years ago
2
add BLEURT and BERTScore
#11
sebastianGehrmann
closed
3 years ago
0
ModuleNotFoundError: No module named 'nltk'
#10
MirunaClinciu
closed
3 years ago
1
ModuleNotFoundError: No module named 'nltk'
#9
MirunaClinciu
closed
3 years ago
2
Script is language-specific, even though GEM isn't
#8
evanmiltenburg
closed
3 years ago
1
Use correct language
#7
tuetschek
closed
3 years ago
1
Add references files for GEM datasets
#6
tuetschek
closed
3 years ago
2
Add Meteor Java checks & download
#5
tuetschek
closed
3 years ago
1
Previous
Next