search

microsoft / glge

Code for ACL2021 paper: "GLGE: A New General Language Generation Evaluation Benchmark"
Other
58 stars 7 forks source link

Reproduce the result of LSTM on PersonaChat #10

Open StevenTang1998 opened 3 years ago

StevenTang1998 commented 3 years ago

Hi, I followed the instruction from this page. I tested the result of the LSTM model on easy PersonaChat dataset. I used the original script to preprocess the dataset combined with the vocab.txt in this page (because I can't find it in GLGE page).

./preprocessed.sh personachat easy

And then I used the original scipt to train and evaluate the lstm model without modifying anything.

./run.sh personachat easy lstm test

However, there is a big gap between the results of mine and the ones in the paper. The results of paper are 42.2/35.9/0.2/0.7. The results of mine are 31.75/19.63/0.3/1.2.

So, I want to know how to get the results in the paper or where I am wrong.

araloak commented 3 years ago

Hi, are you aware how to get the test.tgt file? I downloaded the GLGE dataset on this page, but the test.tgt file is not included.