Balanced CIFAR10 Dataset

[jmduarte]

• Adding balanced CIFAR10 dataset for final performance (code to get indices + index numpy array file)
• With this balanced dataset, the accuracy is higher (and more similar to the accuracy on the full dataset)

Model	Datset	Accuracy
Pretrained ResNet	Full test set	87.2%
Pretrained ResNet	Original subset	85.5%
Pretrained ResNet	Balanced subset	87.0%
Pretrained ResNet TFLite quantized	Full test set	87.0%
Pretrained ResNet TFLite quantized	Original subset	86.5%
Pretrained ResNet TFLite quantized	Balanced subset	87.0%

@cskiraly @maltanar @jeremy-syn

[github-actions[bot]]

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

[maltanar]

Using PERF_SAMPLE=True and QUANT_MODEL=True in tflite_test.py (which were the default values) I was able to reproduce the 87.0% accuracy with the new balanced subset.

Very minor/unrelated-to-PR comment: README specifies # Prepare Python venv (Python 3.6+ and pip>20 required) but scipy==1.6.0 in requirements.txt in fact requires Python 3.7+ as specified here so I had to install that for the reproduction:

https://scipy.github.io/devdocs/dev/toolchain.html#numpy

[maltanar]

I've also signed the CLA now.

[cskiraly]

@jmduarte can you also sign the CLA (or let me know if you have already signed) Thanks

[jmduarte]

@jmduarte can you also sign the CLA (or let me know if you have already signed) Thanks

@cskiraly I have signed the CLA, I think the job is failing from @maltanar. Maybe he needs to link his GitHub to the CLA appropriately?

Created a committerMap for users in PR: (signed: jmduarte-4932543, ) , (notSigned:  maltanar-1880902, )

[cskiraly]

@jmduarte @maltanar , what do you think, should we also update the performance threshold in https://github.com/mlcommons/tiny/blob/master/benchmark/README.md and in https://github.com/mlcommons/tiny/blob/master/benchmark/MLPerfTiny_Rules.adoc ?