search

mlcommons / inference

Reference implementations of MLPerf™ inference benchmarks
https://mlcommons.org/en/groups/inference
Apache License 2.0
1.19k stars 519 forks source link

ssd-resnet34(1200x1200 pytorch) 'code result' <-> 'document' mAP difference #1143

Closed aimfreshwater closed 2 years ago

aimfreshwater commented 2 years ago

I just downloaded param ssd-resnet34 1200x1200 pytorch at table and ran it.

And result is like this. mAP:19.618%

ssd-resnet34_1200_pytorch_result

The code mAP result is different from mAP accuracy value at table(mAP:0.20) https://github.com/mlcommons/inference/tree/master/vision/classification_and_detection

Is the table result(mAP 0.20) right?

psyhtest commented 2 years ago

It's been validated by many submitters, so seems right. At any rate, this benchmark has been deprecated (v2.0 was the last round where it appeared), so I would not spend time on making it run.

aimfreshwater commented 2 years ago

It's been validated by many submitters, so seems right. At any rate, this benchmark has been deprecated (v2.0 was the last round where it appeared), so I would not spend time on making it run.

Thank you for your reply! I couldn't find any of the next round model information. Could I ask where can I find the next round model information?