search

ChengpengChen / RepGhost

RepGhost: A Hardware-Efficient Ghost Module via Re-parameterization
MIT License
168 stars 17 forks source link

Hi~Great work there! Is INT8 quantization all good? #2

Open WenWeiZhao opened 1 year ago

WenWeiZhao commented 1 year ago

Hi~ Great work there! What I want to ask is whether RepGhost has suffered a serious loss after INT8 quantization? Or how do you solve quantitative problems? Thanks~

ChengpengChen commented 1 year ago

Nice question! As we all know, a re-parameterized model is less quantization-friendly than a regular one. In our work, we have not evaluated the quantization performance of RepGhostNet yet. It would be interesting to work on this.

On the other hand, we note that:

  1. (INT8)Quantization of light-weight CNNs on mobile devices may not brings as much speedup as that of large CNNs on GPU or NPU, because operations on fp16/fp32 on mobile devices have been optimized well nowadays, and thus operations on int8 would not bring that much speedup.
  2. Quantization of light-weight CNNs is also difficult, mainly because of the depthwise convolutional layers. Channel-wise or even mixed-precision quantizations are often applied to compensate the quantization loss introduced by depthwise conv. And many other advanced quantization algorithms have been proposed in the literature. RepGhostNet also applies re-parameterization on depthwise conv, we believe that the quantization problem encountered by RepGhostNet would not be much different from other light-weight CNNs.