search

NVIDIA / TensorRT

NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
https://developer.nvidia.com/tensorrt
Apache License 2.0
10.83k stars 2.14k forks source link

stable diffusion quantization in inpainting task is poor #4212

Open worhar opened 1 month ago

worhar commented 1 month ago

i have completed stable diffusion quantization in txt2img as demo shows. the result is very good. when i want to transfer sd quantization in inpainting task, i meet the problem that the quantization result is very poor. the mask area is still noise and the quantized unet model can not inpaint any thing. Do you konw the problem and anyone can help me?

worhar commented 1 month ago

add : My fake quantization results are good

lix19937 commented 4 weeks ago

How about the fp16 performance ? How about ptq ?

yuanyao-nv commented 3 weeks ago

Please also provide more repro information, such as the ONNX model that was used. Thanks.