mlcommons / inference_policies

Issues related to MLPerf™ Inference policies, including rules and suggested changes
https://mlcommons.org/en/groups/inference/
Apache License 2.0
55 stars 52 forks source link

Clarification on reading Quality Target #266

Closed bitfort closed 1 year ago

bitfort commented 1 year ago

Basic question but important.

https://github.com/mlcommons/inference_policies/blob/master/inference_rules.adoc#411-constraints-for-the-closed-division Resnet50-v1.5 "99% of FP32 (76.46%)"

How is the quality target to be read? (A) [0.99] [FP32 (76.46%)] = 75.695% (B) [0.99 FP32] = 76.46%

Based on my understanding going through previous log files, (A) appears to be the correct reading but want to confirm here.

rnaidu02 commented 1 year ago

A is correct (75.695%)

bitfort commented 1 year ago

Thanks!