yizhongw / Tk-Instruct

Tk-Instruct is a Transformer model that is tuned to solve many NLP tasks by following instructions.
https://arxiv.org/abs/2204.07705
MIT License
177 stars 27 forks source link

About the Evaluation Metrics #14

Open albertwy opened 1 year ago

albertwy commented 1 year ago

Hi, your work is very cool and i really like it. I have a question about your adopted evaluation metrics. Using compute_metrics.py, the Exact Matching(EM) and ROUGE-L scores are both reported. But only the ROUGE-L results are reported in the paper. I am confused about it. Could you give me some insights about it? Thanks.

Results reported by compute_metrics.py

image

Results reported by the paper

image