Combination of pruning and INQ

[DAVIDNEWGATE]

Dear Aojun, I have tried to implement your INQ following the paper, and it works pretty good. However, when I combine it with DNS or iterative pruning, the accuracy loss become large. I use the lager absolute value part to form quantization base, the partition is (0.2 0.4 0.6 0.8 1). It is on densenet. Do you have any insight for that case? Kind regards.

[AojunZhou]

the 5 bits quantization? the sparse model partition must be careful, such as 0.05, 0.1,0.15,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9,0.95,1.0. and more training epochs is useful.

[DAVIDNEWGATE]

Thanks, I still don't really understand how to combine it with DNS, I have just done INQ after DNS, there is no difference with just using INQ

[AojunZhou]

yeah, I also just do INQ after DNS, only fixed the zero and using INQ to none-zero part.

[DAVIDNEWGATE]

If we just fixed zero, will retrain recover the accuracy back to the level before INQ? Does that means we should only prune small part on pruning stage? If not, how did you finally get such high compression rate with high accuracy?

[YaoAnbang]

To combine DNS and INQ, you need first run DNS to achieve target sparse rate (e.g., ~18X compression on AlexNet without accuracy loss). Then, you can run INQ over the final model from DNS while keeping zero values fixed (i.e., they are removed from the network). In our ICLR paper, we achieved such an impressive accuracy just in the above way. We have not tested the combination of running DNS and INQ simultaneously, as we clearly clarified in the paper.

[DAVIDNEWGATE]

Thank you, I have tried 0.025 pace after quantization of 90%, the result is amazing. Another issue is that how do you deal with BN layers of resnet? Did you quantize them too?

[AojunZhou]

No, we don't quantize BN layers to low-precision, In my code, you must skip the bn layers.

[KangolHsu]

@Zhouaojun @DAVIDNEWGATE How to skip bn layers （include batchnorm layer and scale layer）？I don't know what exactly you mean。

[AojunZhou]

@KangolHsu you can skip the bn layers (include batch norm and scale layer), you can modify line 764 in "src/caffe/net.cpp" , we have added some comments in "src/caffe/net.cpp"

[KangolHsu]

thanks https://github.com/Zhouaojun/Incremental-Network-Quantization/blob/0548e144898ff5d6d6906ef7ea862bf5e01989f9/src/caffe/net.cpp#L763

[zlheos]

@Zhouaojun @DAVIDNEWGATE why the weights is not 2^(-n) after INQ ? I don't know why?

[AojunZhou]

@zlheos have you completed all quantization step ?

[zlheos]

@Zhouaojun I try to test lenet network , and complete all quantization step then , I find those weights value is not 2^(-n) I feel mask does not effect

[wellhao]

@Zhouaojun @zlheos Have you solved this problem？ I got the same problem. I guess the structure of lenet network is too simple.

[zlheos]

@wellhao I train on tensorflow , and give right answer

[wellhao]

@zlheos could you give me some suggest？thank you！

[litingsjj]

@KangolHsu ，Hi，can you give me some advice about how to modify bn layers?

[zlheos]

@wellhao I train on Tensorflow you can reference this repository https://github.com/DAVIDNEWGATE/Project