search

xuke225 / EQ-Net

EQ-Net [ICCV 2023]
23 stars 3 forks source link

Nice Work ! Can you provide the hyperparameters configuration for training MobileNetV2? #1

Open LuletterSoul opened 11 months ago

LuletterSoul commented 11 months ago

I changed the "arch" in the config to "mobilenetv2" and the number of cards to 8. After training for 79 epochs, the accuracy is only around 40+%.

[meter.py: 338]: [train_epoch] Epoch[79]:   lr: 0.0003  Loss: 1.7097    Top1_acc: 46.03%    Top5_acc: 69.70%    Time: 2817.44s
[meter.py: 338]: [train_epoch] Epoch[79]:   lr: 0.0003  Loss: 1.7141    Top1_acc: 45.84%    Top5_acc: 69.62%    Time: 2817.38s
[meter.py: 338]: [train_epoch] Epoch[79]:   lr: 0.0003  Loss: 1.7137    Top1_acc: 45.85%    Top5_acc: 69.73%    Time: 2817.13s
[meter.py: 338]: [train_epoch] Epoch[79]:   lr: 0.0003  Loss: 1.7210    Top1_acc: 45.65%    Top5_acc: 69.72%    Time: 2817.51s
[meter.py: 338]: [train_epoch] Epoch[79]:   lr: 0.0003  Loss: 1.7124    Top1_acc: 46.03%    Top5_acc: 69.72%    Time: 2816.38s
[meter.py: 338]: [train_epoch] Epoch[79]:   lr: 0.0003  Loss: 1.7149    Top1_acc: 45.76%    Top5_acc: 69.67%    Time: 2816.46s
[meter.py: 338]: [train_epoch] Epoch[79]:   lr: 0.0003  Loss: 1.7089    Top1_acc: 45.99%    Top5_acc: 69.88%    Time: 2815.92s
[meter.py: 338]: [train_epoch] Epoch[79]:   lr: 0.0003  Loss: 1.7139    Top1_acc: 45.80%    Top5_acc: 69.62%    Time: 2816.36s

What mistakes did I make?

HanLeI187 commented 11 months ago

I'm pleased that you're following our work, in mobilenetv2 we train with a quantization bit width of 3bit-8bit, unlike the 2bit-8bit of the resnet series, due to the severe weight competition problem caused by separable convolution.

LuletterSoul commented 11 months ago

@HanLeI187

I'm pleased that you're following our work, in mobilenetv2 we train with a quantization bit width of 3bit-8bit, unlike the 2bit-8bit of the resnet series, due to the severe weight competition problem caused by separable convolution.

Do you mean that I only need to change the bit range of w_bit_list and a_bit_list to [3, 4, 5, 6, 7, 8] ?

HanLeI187 commented 11 months ago

Yes, it is.

liuyiming199721 commented 1 month ago

@HanLeI187

很高兴您关注我们的工作，在mobilenetv2中我们采用3bit-8bit的量化位宽进行训练，与resnet系列的2bit-8bit不同，这是由于可分离卷积引起的严重的权重竞争问题。

你的意思是我只需要将 w_bit_list 和 a_bit_list 的位范围改为 [3, 4, 5, 6, 7, 8] 吗？

估计您跑通了所以没回复啊哈哈哈