Closed wac81 closed 5 years ago
You should run your model on the test data in the json_data folder and let your model generate multiple hypotheses. The test_mult_ref data is aligned with the test data set. Then you can have your own script to compute the score against the reference vs hypotheses.
Hi,
I want to use the test dataset with multiple references(which you uploaded at https://github.com/snakeztc/NeuralDialog-CVAE/blob/master/data/test_multi_ref.json.) to get the evaluation result you presented in the paper. But I'm pretty confused because the format of this one is quite different from the one with single reference, for it's in lack of some information, such as topic. Could you please tell me how to get the evaluation result with multiple references with this test dataset which seems to be incomplete?
Looking forward to your reply. :)