search

tuetschek / e2e-metrics

E2E NLG Challenge Evaluation metrics
http://www.macs.hw.ac.uk/InteractionLab/E2E/
Other
91 stars 22 forks source link

The program gets stuck at "Running Py-MTEval metrics..." #3

Open genewliu opened 1 year ago

genewliu commented 1 year ago

Results (2022-11-11).txt

The program gets stuck at "Running Py-MTEval metrics..."

Running MS-COCO evaluator... creating index... index created! Loading and preparing results...
DONE (t=0.00s) creating index... index created! tokenization... PTBTokenizer tokenized 132573 tokens at 458797.38 tokens per second. PTBTokenizer tokenized 18181 tokens at 124252.45 tokens per second. setting up scorers... computing METEOR score... METEOR: 0.452 computing Rouge score... ROUGE_L: 0.699 computing CIDEr score... CIDEr: 2.379 Running Py-MTEval metrics...

genewliu commented 1 year ago

Even here it ended at the same line: https://worksheets.codalab.org/bundles/0x527d31b09b464563b73e45300ee405b2

tuetschek commented 1 year ago

@genewliu I'm sorry, I'm not really able to work much on the package at the moment (and it might be better to use Huggingface, Torchmetrics, SacreBLEU, GEM-metrics). What data are you working on, are you able to run it on the example inputs?