Resume calibrated models

[mason5957]

Hello,

I have recently been trying to resume an ImageNet model and came across a question. According to the code, when the --resume command is used, it seems that if the --quant_act is also specified, it needs to be executed again as indicated by line 616. I would like to know if this is necessary?

I have also tried not to run pd_optimize_timeembed and pd_optimize_timewise, but the results were somewhat abnormal.

Thank you.

[hatchetProject]

Hi, --resume is for resuming the whole model with both weight and activation quantized. Line 616 does not need to be executed again. You can comment them after the first run.

It seems the images produced before running _pd_optimizetimeembed and _pd_optimizetimewise are corrupted. You can verify by checking the code step-by-step (inference with only-weight-quantized by using set_quant_state(True, False), inference with weight-and-activation-quantized). If you are doing W4A4, then please exclude the --running_stat argument.

[mason5957]

Thanks, i will try it.

[mason5957]

@hatchetProject Hello, I've attempted to inference with only-weight-quantized by using set_quant_state(True, False), but it seems like the result is still showing abnormalities. Would you mind trying to quantize ImageNet into W4A8 and checking if there are any errors in the weight quantization process? Thank you for your assistance!

Here is my scripts: python sample_diffusion_ldm_imagenet.py --resume_base models/ldm/cin256-v2/model.ckpt --n_samples 25 --batch_size 25 --custom_steps 20 --eta 0.0 --seed 3407 --ptq --quant_act --quant_mode qdiff --cali_st 20 --cali_batch_size 16 --cali_n 64 --weight_bit 4 --act_bit 8 --device cuda:0 --a_sym --a_min_max --running_stat --cond -l result --cali_data_path XXX.pth --name quest_cali_n_64 --verbose --cali_iters 10000

[hatchetProject]

I think the images are only a little dark, but other details are good and you don't need to worry about it. This might be relevant to the random seed you use (seed=3407). Also, the weight quantization process won't be the critical bottleneck, and we actually did not quantize weights carefully in the experiments.

What I recommend is to inference 10k/50k images and calculate the fid/sfid to evaluate the performance and see whether everything is fine.