Closed ianz27 closed 1 year ago
@ZQ-zz ,
There is no model evaluation after fast-finetine. You should write the code like:
quant_mode = 'calib'
quantizer = torch_quantizer(quant_mode=quant_mode,
module=model,
input_args=quant_input,
# output_dir=quant_output_dir,
bitwidth=self.quant_bitwidth,
device=quant_device)
quant_model = quantizer.quant_model
# print('quant_model type:', type(quant_model))
if self.fast_finetune:
quantizer.fast_finetune(ft_run_fn, (quant_model, self.dataloader, self.ft_max_num))
ft_run_fn(quant_model, self.dataloader, self.ft_max_num)
quantizer.export_quant_config()
for name, m in quant_model.named_children():
print(name)
try:
print('m.quantizer.bias_corr[m.node.name]: ', m.quantizer.bias_corr[m.node.name])
Hi I was successd in PTQ but failed with fast-finetune the code is like below:
when enable fast-finetune, bias_corr is None
How to solve the error, as there is on error in the fast-finetune workflow