ModelTC / MQBench

Model Quantization Benchmark
Apache License 2.0
769 stars 140 forks source link

Can't convert to deploy after use advanced_ptq #262

Closed buzhou1 closed 1 year ago

buzhou1 commented 1 year ago

I try to use advanced ptq according to example file(imagenet) but it cant convert to deploy because model.inits didn't realize the node named number only like the picture. How to solve ` if opt.quantize: prepare_custom_config_dict= { 'extra_qconfig_dict':{'w_observer': 'MinMaxObserver', 'a_observer': 'EMAMinMaxObserver', 'w_fakequantize': 'AdaRoundFakeQuantize', 'a_fakequantize': 'PACTFakeQuantize'},
} reconstruction={ 'pattern': 'block', 'scale_lr': 4.0e-5, 'warm_up': 0.2, 'weight': 0.01, 'max_count': 20000, 'b_range': [20,2], 'keep_gpu': True, 'round_mode': 'learned_hard_sigmoid', 'prob': 0.5, }

    model.eval()
    model = prepare_by_platform(model, backend, prepare_custom_config_dict)
    model = model.to(device)

    calibration_batchsize = 4
    cali_data=[]
    # calibration loop
    for i, (imgs, targets, path, _) in enumerate(dataloader):
        cali_data.append(imgs)

    # do activation and weight calibration seperately for quick MSE per-channel for weight one
    model.eval()
    with torch.no_grad():
        enable_calibration_woquantization(model, quantizer_type='act_fake_quant')
        for batch in cali_data:
            model(batch.cuda())
        enable_calibration_woquantization(model, quantizer_type='weight_fake_quant')
        model(cali_data[0].cuda())
    print('begin advanced PTQ now!')
    model = ptq_reconstruction(model, cali_data, reconstruction)
    enable_quantization(model)

    model.eval()
    convert_deploy(model, backend.HIPU, input_shape, output_path='./onnx', model_name=opt.output_name )
    output_path = './onnx'
    print(f'ONNX saved {output_path} /{opt.output_name}')`
buzhou1 commented 1 year ago

this is my onnx and the error image line 645, in replace_expand_to_intexpand init = model.inits[scale] KeyError: '2861'

github-actions[bot] commented 1 year ago

This issue has not received any updates in 120 days. Please reply to this issue if this still unresolved!