For Resnet-50 "System energy (J) per stream*", there are only 2 digits of precision and there are many submissions at the lowest end of precision, e.g. "0.01" and "0.02" and "0.04".
It's hard to differentiate submissions with this lack of precision. To understand these submissions I had to go back to the raw logs and parse out the power values myself.
A few fixes might be, for example,
Use mJ instead of J (i.e. multiply all system energy numbers numbers by 1000)
Add a 3rd (& 4th?) decimal place
Use a specific number of significant digits, as opposed to fixed point
Look at the results table;
"Closed Power" Tab in https://mlcommons.org/en/inference-edge-21/
For Resnet-50 "System energy (J) per stream*", there are only 2 digits of precision and there are many submissions at the lowest end of precision, e.g. "0.01" and "0.02" and "0.04".
It's hard to differentiate submissions with this lack of precision. To understand these submissions I had to go back to the raw logs and parse out the power values myself.
A few fixes might be, for example,
What are our options here?