Open raihankhan-rk opened 1 year ago
I think it should be compute_perplexity.py
Also, could someone provide a good guide on how to run this model locally on PC?
This isn’t a language model. It’s a utility for doing analysis of language models. There’s a very good chance that this isn’t what you’re looking for.
I want to compute bpb for language models, what is this argument -- utf8_conversion_scalar ?
I can't seem to find the 'run_lm_perplexity.py' file. Can someone help, please?
Also, could someone provide a good guide on how to run this model locally on PC?