Could you please explain more about Integrity evaluation?

lipiji / SongNet

Code for ACL 2020 paper "Rigid Formats Controlled Text Generation"：https://www.aclweb.org/anthology/2020.acl-main.68/

MIT License

230 stars 40 forks source link

Could you please explain more about Integrity evaluation? #19

Closed DuYooho closed 3 years ago

DuYooho commented 3 years ago

I cannot find the codes about evaluating Integrity. Could you please release them as well as the pretrained GPT2 model mentioned in your paper? Thanks.

lipiji commented 3 years ago

@DuYooho You may regard Integrity as the ppl of punctuation symbols. So after obtaining the generated results I used a LM to calculate the score. https://github.com/lipiji/Guyu