mlcommons / inference

Reference implementations of MLPerf™ inference benchmarks
https://mlcommons.org/en/groups/inference
Apache License 2.0
1.19k stars 519 forks source link

Run image classification benchmark on real imageNet2012 dataset #1404

Closed manyiw99 closed 1 year ago

manyiw99 commented 1 year ago

Hi, since you haven't provided preprocess steps of imagenet2012 validation dataset for vision/classfication and detection benchmark. And I hope to run the benchmark on this real dataset. In your Getting started tutorial, you use make_fake_imagenet.sh script to generate a val_map.txt file. I was wondering, if I hope to use the real imagenet2012 validation dateset, do I need to generate a similar val_map.txt as well?

If so, does the second column of this file mean the category of the image? I have downloaded the dataset, however, it seems that it only includes a bunch of images without labels. So I was wondering how should I get the category if the second column does mean the category?

Thank you very much for your help!

Here's the val_map.txt generated by make_fake_imagenet.sh (Originial includes 8 rows, here I pasted only 2 rows of the file):

val/800px-Porsche_991_silver_IAA.jpg 817
val/512px-Cacatua_moluccensis_-Cincinnati_Zoo-8a.jpg 89
arjunsuresh commented 1 year ago

Is this what you're looking at?

For using the full dataset we need to get imagenex-aux in addition to the imagenet-val dataset.

manyiw99 commented 1 year ago

Yes, the links are very helpful. Thank you!