MSTTR tests fail with faulty results

GEM-benchmark / GEM-metrics

Automatic metrics for GEM tasks

https://gem-benchmark.com

MIT License

60 stars 20 forks source link

MSTTR tests fail with faulty results #55

Closed ndaheim closed 2 years ago

ndaheim commented 2 years ago

test_msttr_disjoint_tokens fails with: AssertionError: 0.95455 != 1 within 7 places (0.04544999999999999 difference) test_msttr_identical_tokens fails with: AssertionError: 0.39394 != 1.0 within 7 places (0.60606 difference)

ndaheim commented 2 years ago

Should be fixed by #70 , so I'll close this.