search

bytedance / MRECG

Apache License 2.0
35 stars 4 forks source link

Does it necessary to run weight quantization reconstruction before full quantization reconstruction? Or it is independent? #8

Open padeirocarlos opened 6 months ago

padeirocarlos commented 6 months ago

If my target is to get a full quantization model, does it necessary to do weight quantization reconstruction before full quantization reconstruction as the shown?

Can I skip weight quantization reconstruction and turn on full quantization reconstruction directly!

BobxmuMa commented 6 months ago

Sorry for not seeing the message in time.

BRECQ [1] reconstructs weights and activations separately. However, a direct full quantization reconstruction has similar accuracy to a separate reconstruction.

Best wishes.

[1] Yuhang et al. BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstruction

padeirocarlos commented 6 months ago

A simple clarification. I noted that you do not initialize the weight quantizer parameters. Why? When I tried to initialize it took so long to run. Is that why you do not initiliaze? Correct me if I am wrong please.

BobxmuMa commented 6 months ago

A simple clarification. I noted that you do not initialize the weight quantizer parameters. Why? When I tried to initialize it took so long to run. Is that why you do not initiliaze? Correct me if I am wrong please.

Because direct full quantization without first quantizing the weights reproduces BRECQ accuracy similar to the original paper. In addition, the more efficient direct full quantization makes prior quantization of weights unnecessary when accuracy is not affected.

padeirocarlos commented 6 months ago

A simple clarification. As referenced by you in this paper [1] Yuhang et al. BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstruction.

I noted that they do not initialize the weight quantizer parameters. Why? When I tried to initialize it took so long to run. Here what I am saying do not initialize.

self.inited variable start as FALSE self.inited = False https://github.com/yhhhli/BRECQ/blob/main/quant/quant_layer.py#L54

Here the self.inited variable must be FALSE to be initialized https://github.com/yhhhli/BRECQ/blob/main/quant/quant_layer.py#L61

It improve the accuracy if it is initialized? Or the accuracy is the same?

BobxmuMa commented 6 months ago

A simple clarification. As referenced by you in this paper [1] Yuhang et al. BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstruction.

I noted that they do not initialize the weight quantizer parameters. Why? When I tried to initialize it took so long to run. Here what I am saying do not initialize.

self.inited variable start as FALSE self.inited = False https://github.com/yhhhli/BRECQ/blob/main/quant/quant_layer.py#L54

Here the self.inited variable must be FALSE to be initialized https://github.com/yhhhli/BRECQ/blob/main/quant/quant_layer.py#L61

It improve the accuracy if it is initialized? Or the accuracy is the same?

BRECQ has initialized quantizer parameters. Calibration is necessary in BRECQ. The quantization after calibration is static quantization. L194-L195