SNPE correctness problem

XiaoMi / mobile-ai-bench

Benchmarking Neural Network Inference on Mobile Devices

Apache License 2.0

362 stars 58 forks source link

SNPE correctness problem #29

Closed ysh329 closed 5 years ago

ysh329 commented 5 years ago

Hi, all,

I recently tested SNPE according its doc not mobile-ai-bench, and found its results are wrong for mobilenetv1, v2 on Snapdragon 835 (check using its Python scriptmodels/alexnet/scripts/show_alexnet_classifications.py).

Thus, I wanna ask do you verify the correctness? Thanks in advance.

lee-bin commented 5 years ago

I'm not familiar with models/alexnet/scripts/show_alexnet_classifications.py. What did you test? If you used SNPE tools to quantize MobileNetV1 or MobileNetV2, accuracy should be very low. But other than that, accuracy is pretty close to the original model.

ysh329 commented 5 years ago

I tested mobilenet and its accruacy of quantitzed mobilenet indeed are wrong. Thanks~