Quantized Accuracy of Advanced-ptq is lower than that of Naive-ptq for YOLOv5?

[xingyueye]

Hi~ I have quantized YOLOv5s models with naive-ptq and advanced-ptq respectively. The mAP of basemodel is 36.3, while naive-ptq is 35.8 and advanced-ptq is 35.0. The quantized results of advanced-ptq is worse than Naive-ptq. The setting of advanced-ptq as below:

extra_prepare_dict:
    extra_qconfig_dict:
        w_observer: MSEObserver
        a_observer: EMAMSEObserver
        w_fakequantize: AdaRoundFakeQuantize
        a_fakequantize: QDropFakeQuantize
        w_qscheme:
            bit: 8
            symmetry: True
            per_channel: True
            pot_scale: False
            p: 2.4
        a_qscheme:
            bit: 8
            symmetry: True
            per_channel: False
            pot_scale: False
            p: 2.4
quantize:
    quantize_type: advanced_ptq # support naive_ptq or advanced_ptq
    cali_batchsize: 16
    reconstruction:
        pattern: block
        scale_lr: 4.0e-5
        warm_up: 0.2
        weight: 0.01
        max_count: 20000
        b_range: [ 20,2 ]
        keep_gpu: True
        round_mode: learned_hard_sigmoid
        prob: 0.5

Please help with how to get a better advanced-ptq results.

[PannenetsF]

'cali_batchsize: 16' means the mqbench will use 16 batches to calibrate the model. If you have set your batch size to 4 and 4 gpus are used, there are 16x4x4=256 images used.

Please check the number of image used in calibration. I recommend 256 at least.

[xingyueye]

Hi~ I found that when I increase the 'cali_batchsize', the adv-ptq process is easily to broke down with the error of CUDA out of Memory. I'm confused about how the calib_data is allocated during adv_ptq process?

[PannenetsF]

@xingyueye set keep_gpu to False.

Sorry for such a late reply, as I was on vacation.

[wangshankun]

@PannenetsF same issue，although set keep_gpu to False

[github-actions[bot]]

This issue has not received any updates in 120 days. Please reply to this issue if this still unresolved!