search

rpryzant / delete_retrieve_generate

PyTorch implementation of the Delete, Retrieve Generate style transfer algorithm
MIT License
132 stars 26 forks source link

Which language model is trained to calculate the LPP ? #28

Closed naveen-kinnal closed 3 years ago

naveen-kinnal commented 3 years ago

@rpryzant From the paper, I see that a separate language model is trained to compute the perplexity. However, I am not able to find that in the code explicitly. All I can see is, LPP is calculated as the mean of the 'dev_loss'. Could you please tell, which language model is being trained?

rpryzant commented 3 years ago

Here's how it's calculated https://github.com/rpryzant/delete_retrieve_generate/blob/79bf90ed0fe772061da54b32486bd02771dc3348/src/evaluation.py#L160

It's the current model's loss over the dev set -- using a separate language model would be a different evaluation procedure and you are welcome to contribute it if you want!

rpryzant commented 3 years ago

Closing due to inactivity.