search

qywu / TextGAIL

MIT License
28 stars 15 forks source link

Question about evaluation metrics #1

Closed shizhediao closed 3 years ago

shizhediao commented 4 years ago

Hi authors, Thanks for your great work titled "TextGAIL: Generative Adversarial Imitation Learning for Text Generation". I have a question about your experiment metrics. The paper stated that "When using BLEU for unconditional generation tasks, the entire training corpus is used as references for BLEU". I was wondering what is the justification of using training corpus instead of test corpus? I looked up SeqGAN and other text+GAN papers but do not find reasons that can support this statement. Perhaps I omit the detail, could you point it out?

Thanks!

Angelina7329 commented 3 years ago

Hello, authors, thank you for the open source code. I found that the version of the ‘torchfly’ library used in your code has changed. Do you still keep the previous open source version of the library?

qywu commented 3 years ago

Hi Angelina7329, thank you for your question. I have updated TextGAIL in the past months. I will upload a new version soon that is compatible with the existing codes.