search

mlcommons / inference_policies

Issues related to MLPerf™ Inference policies, including rules and suggested changes
https://mlcommons.org/en/groups/inference/
Apache License 2.0
55 stars 52 forks source link

Specification for Language Modelling #293

Open jubueche opened 6 months ago

jubueche commented 6 months ago

Hi,

I see many criterions for accuracy, for example the accuracy must be within 99% of the FP-32 accuracy. Is there a similar idea for perplexity scores in language modelling?