open-mmlab / mmrazor

OpenMMLab Model Compression Toolbox and Benchmark.
https://mmrazor.readthedocs.io/en/latest/
Apache License 2.0
1.47k stars 228 forks source link

the gap between your realization and the paper in autoslim #64

Closed priscillapan closed 2 years ago

priscillapan commented 2 years ago

In the paper, AutoSlim-MobileNet-v2 at 305M FLOPs achieves 74.2% top-1 accuracy, 2.4% better than default MobileNet-v2 (301M FLOPs),but your realization is only 72.73% at 320M FLOPs。

What do you mean by "Note that we ran the official code and the Top-1 Acc of the models with official channel cfg are 73.8%, 72.5% and 71.1%"? It should be 74.2%. I test it using paper's official code and official channel cfg(https://github.com/JiahuiYu/slimmable_networks). The result is 74.2% not 72.5%

pppppM commented 2 years ago

This result is obtained by retraining with the official code, not the test result using the official checkpoint.

priscillapan commented 2 years ago

I see. Thanks. I‘ll try it later. Now I'm training subnet at 200M Flops by mmrazor :)