search

Alex-Fabbri / Multi-News

Large-scale multi-document summarization dataset and code
Other
274 stars 53 forks source link

Training and validation accuracy of models #22

Closed martin-kirilov closed 4 years ago

martin-kirilov commented 4 years ago

Hi Alex,

I've been training the transformer model from OpenNMT on your dataset, as explained in another issue. I was wondering if I am doing anything wrong, since the training accuracy is around 44, and the validation accuracy is around 37 at step 20000. Do you know what are the accuracies of the models you provided (e.g. I can see that your transformer model is also trained for 20000 steps)?

martin-kirilov commented 4 years ago

I have also generated some summaries using the two models (my transformer at 20000 steps and your pre-trained model), and they are quite different in terms of quality and factual accuracy.