Closed 18582088138 closed 4 months ago
Hi @18582088138, thanks a lot for reporting the issue and apologies for the delay. This example was deprecated and the new methodology recommended by the NNCF team is now hybrid quantization (which will be available in optimum-intel v1.16.0 but you can install optimum-intel from source in the meantime) where the model's weights are quantized, and only the activations of the U-Net component.
from optimum.intel import OVStableDiffusionPipeline, OVWeightQuantizationConfig
model = OVStableDiffusionPipeline.from_pretrained(
model_id,
export=True,
quantization_config=OVWeightQuantizationConfig(bits=8, dataset="conceptual_captions"),
)
For additional information you can also take a look at this notebook or directly to our documentation
Reference the method in OpenVINO Stable Diffusion QAT example , the original sample code has already run success, I find that in this way, the unet a convert by INT8 pytorch -> INT8 ONNX -> INT8 IR, so that I try to using pytorch FrountEnd to directly convert INT8 pytorch to INT8 IR. but then
`
def export_unet_pytorchFE(export_unet, save_dir):
inputs = {
"sample": torch.ones((2,4,64,64)),
"timestep": torch.tensor(1),
"encoder_hidden_states": torch.randn((2,77,768)),
}
input_names = ["sample", "timestep", "encoder_hidden_states"]
UNET_CONTROL_OV_PATH = Path(f"{save_dir}/unet_int8.xml")
unet = export_unet
unet.eval().cpu()
convert_model
the compressedexport_unet
model , the whole pipeline will hang. I guess this problem may be caused by pytorch FE's inability to parse the INT8 pytorch model. I hope to get your confirmation and want to know the cause of this problem. The following is thepytorch FE model convert
part source code.The log before pipeline hang is in following:![image](https://github.com/huggingface/optimum-intel/assets/26618683/cac87ca7-da6d-4794-8496-e442dc902d8d)
My device environment info:
Thanks for your attention.