issues
search
GEM-benchmark
/
GEM-metrics
Automatic metrics for GEM tasks
https://gem-benchmark.com
MIT License
60
stars
20
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Installation errors -- Cython and Questeval
#104
ashleylew
closed
9 months ago
1
Issues generating Heavy metrics
#103
SimonvdFliert
opened
1 year ago
0
Troubles running embedding metrics in Colab (BERTscore, BLEURT and Prism)
#102
asnota
opened
1 year ago
2
What _is_ the "checkout"?
#101
thesofakillers
closed
1 year ago
2
Bugfix for multi-ref: allow bare lists of lists
#100
tuetschek
closed
1 year ago
0
Colab Predictions() and References() instances initialization problem
#99
asnota
closed
1 year ago
6
Modified fix for `references` vs. `targets`.
#98
tuetschek
closed
2 years ago
1
Use multi-references
#97
jordiclive
closed
2 years ago
1
Key hyperparameters to reproduce results？
#96
VickiCui
opened
2 years ago
3
Rename Hugging Face Hub path
#95
lewtun
closed
2 years ago
0
Better docs & better use as a library
#94
tuetschek
closed
2 years ago
1
Metric versioning
#93
jordiclive
opened
2 years ago
6
Check for reference data type?
#92
tuetschek
closed
2 years ago
2
MoverScore
#91
jordiclive
closed
2 years ago
3
Adding WER (Word Error Rate)
#90
jordiclive
closed
2 years ago
1
CIDEr
#89
jordiclive
closed
2 years ago
1
Support for Multilingual Rouge
#88
ratishsp
opened
2 years ago
0
CUDA out of memory when computing `bleurt` on GEM submission
#87
lewtun
opened
2 years ago
0
Adding Two Diversity Metrics: TTR and Yule's I-Measure
#86
vyraun
closed
2 years ago
2
Support for iBLEU for paraphrasing
#85
ratishsp
opened
2 years ago
0
[WIP] Switching NUBIA implementation to Repro
#84
danieldeutsch
closed
2 years ago
0
Fixing aggregation function
#83
danieldeutsch
closed
2 years ago
1
Adding Translation error rate, TER metric
#82
leoribeiro
closed
2 years ago
4
Adding ability to run on-demand unit tests
#81
danieldeutsch
closed
2 years ago
4
Converting BLEURT implementation to the Repro version
#80
danieldeutsch
closed
2 years ago
2
Adding Prism and generic ReproReferencedMetric base class for Repro metrics
#79
danieldeutsch
closed
2 years ago
3
Adds documentation (#29)
#78
ndaheim
closed
2 years ago
1
Fixes #76: Heavy metrics are not calculated in single-file mode
#77
madaan
closed
2 years ago
0
Heavy metrics are not calculated in single-file mode despite using the heavy-metrics option
#76
madaan
closed
2 years ago
0
#30: add type hints
#75
madaan
closed
2 years ago
0
Add support for docker and add all the metrics from repro
#74
sebastianGehrmann
opened
2 years ago
0
Use reference dataset on the Hugging Face Hub
#73
lewtun
closed
2 years ago
6
MAUVE metric for open-ended text generation
#72
asnota
opened
2 years ago
1
MATTR metric
#71
ndaheim
opened
2 years ago
0
fixes msttr (#55)
#70
ndaheim
closed
2 years ago
0
Change cached_property and union operator for backwards compatibility
#69
j-chim
closed
2 years ago
2
[WIP] Adding Docker version of Prism
#68
danieldeutsch
closed
2 years ago
4
#3 and necessary changes for new sacrebleu version
#67
ndaheim
closed
2 years ago
3
L'Ambre metric
#66
sebastianGehrmann
opened
2 years ago
1
Tests failing
#65
asnota
closed
2 years ago
4
Language key typo
#64
asnota
closed
2 years ago
1
Language key correction
#63
asnota
closed
2 years ago
0
Example on a single metric usage with the input format
#62
asnota
closed
2 years ago
2
Colab or Kaggle compatibility (Python 3.7 usage instead of 3.8)
#61
asnota
closed
2 years ago
1
Document API usage of the library
#60
tuetschek
closed
2 years ago
1
Add a unique versioning key to each metric
#59
sebastianGehrmann
opened
2 years ago
0
Change caching to hash of input/output/source instead of filepath based key
#58
sebastianGehrmann
opened
2 years ago
0
Adds automated testing
#57
ndaheim
closed
2 years ago
0
fixes issue #54 and refactors tests
#56
ndaheim
closed
2 years ago
1
MSTTR tests fail with faulty results
#55
ndaheim
closed
2 years ago
1
Next