Evaluation results

[wangyu0627]

I restart training on the yelp dataset exactly as instructed. Why is it that my local deployment of bertscore gets results so different from yours?

In evaluation/metric.py def BERT_score(predictions, references): bertscore = evaluate.load("bertscore.py") results = bertscore.compute( predictions=predictions, references=references, model_type="roberta-large", num_layers=model2layers["roberta-large"], )

[Martin-qyma]

Thank you for your interest in XRec! You can set rescale_with_baseline=True in bertscore.compute. For more details, please refer to the implementation in evaluation/metric.py.

This operation scales the BERTScore between 0 and 1. It does not affect the ranking ability or the correlation with human judgments, but is intended to improve the readability of the score. We hope you find this helpful.

[wangyu0627]

Thanks, your response was very helpful!