Calculate perplexity on a text with pre-trained language models. Support MLM (eg. DeBERTa), recurrent LM (eg. GPT3), and encoder-decoder LM (eg. Flan-T5).
MIT License
132
stars
11
forks
source link
add `to(self.device)` in `label` to avoid ERROR: tensor on different … #13
…devices when cal loss