Xilinx / Vitis-AI

Vitis AI is Xilinx’s development stack for AI inference on Xilinx hardware platforms, including both edge devices and Alveo cards.
https://www.xilinx.com/ai
Apache License 2.0
1.49k stars 634 forks source link

fast-finetune bias_corr data is None #1168

Closed ianz27 closed 1 year ago

ianz27 commented 1 year ago

Hi I was successd in PTQ but failed with fast-finetune the code is like below:

            quant_mode = 'calib'
            quantizer = torch_quantizer(quant_mode=quant_mode,
                                        module=model,
                                        input_args=quant_input,
                                        # output_dir=quant_output_dir,
                                        bitwidth=self.quant_bitwidth,
                                        device=quant_device)
            quant_model = quantizer.quant_model
            # print('quant_model type:', type(quant_model))

            if self.fast_finetune:
                quantizer.fast_finetune(ft_run_fn, (quant_model, self.dataloader, self.ft_max_num))
                quantizer.export_quant_config()
            else:
                ft_run_fn(quant_model, self.dataloader, self.ft_max_num)
                quantizer.export_quant_config()

            for name, m in quant_model.named_children():
                print(name)
                try:
                    print('m.quantizer.bias_corr[m.node.name]: ', m.quantizer.bias_corr[m.node.name])

when enable fast-finetune, bias_corr is None

deephi_Conv2d(3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
module_0
module_1
m.quantizer.bias_corr[m.node.name]:  None
module_2
module_3
m.quantizer.bias_corr[m.node.name]:  None
module_4
module_5
m.quantizer.bias_corr[m.node.name]:  None
module_6
module_7
module_8
m.quantizer.bias_corr[m.node.name]:  None

How to solve the error, as there is on error in the fast-finetune workflow

[VAIQ_NOTE]: Loading NNDCT kernels...

[VAIQ_NOTE]: OS and CPU information:
               system --- Linux
                 node --- storagex-gpu06
              release --- 5.4.0-132-generic
              version --- #148~18.04.1-Ubuntu SMP Mon Oct 24 20:41:14 UTC 2022
              machine --- x86_64
            processor --- x86_64

[VAIQ_NOTE]: Tools version information:
                  GCC --- GCC 11.2.0
               python --- 3.7.16
              pytorch --- 1.12.1
        vai_q_pytorch --- 3.0.0+a44284e+torch1.12.1

[VAIQ_NOTE]: GPU information:
          device name --- NVIDIA GeForce RTX 3090
     device available --- True
         device count --- 4
       current device --- 0

[VAIQ_NOTE]: Quant config file is empty, use default quant configuration

[VAIQ_NOTE]: Quantization calibration process start up...

[VAIQ_NOTE]: =>Quant Module is in 'cuda'.

[VAIQ_NOTE]: =>Parsing ImageClassifierQ...

[VAIQ_NOTE]: Start to trace and freeze model...

[VAIQ_NOTE]: The input model ImageClassifierQ is torch.nn.Module.

[VAIQ_NOTE]: Finish tracing.

[VAIQ_NOTE]: Processing ops...
██████████████████████████████████████████████████| 67/67 [00:00<00:00, 4319.90it/s, OpInfo: name = return_0, type = Return]                                                                                   

[VAIQ_NOTE]: =>Doing weights equalization...

[VAIQ_NOTE]: =>Quantizable module is generated.(quantize_result/ImageClassifierQ.py)

[VAIQ_NOTE]: =>Get module with quantization.

[VAIQ_NOTE]: =>Preparing data for fast finetuning module parameters ...

[VAIQ_NOTE]: Mem status(total mem: 125.52G, available mem: 108.65G).

[VAIQ_NOTE]: Memory cost by fast finetuning is 0.01 G.

[VAIQ_NOTE]: =>Find initial quantization steps for fast finetuning...

[VAIQ_NOTE]: =>Fast finetuning module parameters for better quantization accuracy...
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:31<00:00,  1.57s/it]

[VAIQ_NOTE]: =>Export fast finetuned parameters ...

[VAIQ_NOTE]: =>Exporting quant model parameters.(quantize_result/param.pth)

[VAIQ_NOTE]: =>Exporting quant config.(quantize_result/quant_info.json)
niuxjxlnx commented 1 year ago

@ZQ-zz ,

There is no model evaluation after fast-finetine. You should write the code like:

            quant_mode = 'calib'
            quantizer = torch_quantizer(quant_mode=quant_mode,
                                        module=model,
                                        input_args=quant_input,
                                        # output_dir=quant_output_dir,
                                        bitwidth=self.quant_bitwidth,
                                        device=quant_device)
            quant_model = quantizer.quant_model
            # print('quant_model type:', type(quant_model))

            if self.fast_finetune:
                quantizer.fast_finetune(ft_run_fn, (quant_model, self.dataloader, self.ft_max_num))

            ft_run_fn(quant_model, self.dataloader, self.ft_max_num)
            quantizer.export_quant_config()

            for name, m in quant_model.named_children():
                print(name)
                try:
                    print('m.quantizer.bias_corr[m.node.name]: ', m.quantizer.bias_corr[m.node.name])