search

awslabs / mlm-scoring

Python library & examples for Masked Language Model Scoring (ACL 2020)
https://www.aclweb.org/anthology/2020.acl-main.240/
Apache License 2.0
333 stars 59 forks source link

How to Compute Perplexity of a Sentence #18

Closed aflah02 closed 2 years ago

aflah02 commented 2 years ago

Hey Is there some way to just use a pre trained model and give a sentence as input and get it's perplexity as the output?

ruanchaves commented 2 years ago

I'm not sure if I understand your question correctly but that is exactly what scorer.score_sentences does ( there is an example on the README ).

aflah02 commented 2 years ago

Thanks I'll check it out!