ZhaolinGao
commented
7 months ago Thank you for your interest in the project. We update the code to include a demo for metric.
- For BLEU score, we follow the typical BLEU setting and use the geometric average of unigram, bigram, trigram, and quadgram which is BLEU-4.
- For Rouge and BERTScore, we use the f1 scores which is a harmonic mean of precision and recall.
- Both specificity and coverability are using the f1 score of BERTScore.
As a novice in NLP, I am not well versed in the metrics used in this article. After consulting relevant materials, I have learned that BLEU comes with variations such as BLEU-1, BLEU-2, BLEU-3, and BLEU-4, which respectively assess the match of n-grams of different lengths. Moreover, Rouge and BERT Score also encompass multiple metrics, including Precision, Recall, and F1 scores. I am unsure about the specific metrics being utilized within the tables presented. Additionally, the author has introduced two novel measures. To facilitate readers' better understanding and assessment of the quality under review, I would appreciate it if the author could provide the corresponding metric calculation codes or more detailed explanations.