search

mlcommons / inference_policies

Issues related to MLPerf™ Inference policies, including rules and suggested changes
https://mlcommons.org/en/groups/inference/
Apache License 2.0
55 stars 52 forks source link

Update llama2 accuracy in the rule #289

Closed nvzhihanj closed 7 months ago

nvzhihanj commented 7 months ago

Address https://github.com/mlcommons/inference/issues/1558

Old accuracy metrics: {'rouge1': 43.88, 'rouge2': 21.7108, 'rougeL': 28.2502, 'tokens_per_sample': 293.3}

New accuracy metrics: {'rouge1': 44.4312, 'rouge2': 22.0352, 'rougeL': 28.6162, 'tokens_per_sample': 294.45}

github-actions[bot] commented 7 months ago

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅