search

salesforce / ctrl

Conditional Transformer Language Model for Controllable Generation
https://arxiv.org/abs/1909.05858
BSD 3-Clause "New" or "Revised" License
1.87k stars 208 forks source link

benchmarking with GPT-2 #47

Open leejason opened 5 years ago

leejason commented 5 years ago

Any suggestion for benchmarking CTRL with GPT-2? Say, loss value, PPL, or any metric to measure text generation quality?

julien-c commented 5 years ago

Not a direct answer to your question, but this (timely) article by @chiphuyen is really good

https://thegradient.pub/understanding-evaluation-metrics-for-language-models/

leejason commented 5 years ago

very helpful & thanks