search

quic / ai-hub-models

The Qualcomm® AI Hub Models are a collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.) and ready to deploy on Qualcomm® devices.
https://aihub.qualcomm.com
BSD 3-Clause "New" or "Revised" License
438 stars 60 forks source link

[BUG] Controlnet models are providing noisy image outputs #37

Open kristoftunner opened 5 months ago

kristoftunner commented 5 months ago

Describe the bug The controlnet pipeline is providing me noisy image output, in general it does its thing, but the quality of the image makes it unsable. Prompt: "photo, mid century modern, natural light, pink walls". See the input/output images below: Input: image

Outputs with various seed/guidance scale settings: image_dpmsolver++_23_2048775510 20_7 5 image_dpmsolver++_23_6126326530 61_7 5 image_dpmsolver++_23_10000000000 00_7 5 image_dpmsolver++_35_2660408163 27_12 5

To Reproduce Try to run the demo application with the following settings:

I get this noise in every image, the image quality is nowhere compared to the original float controlnet model in here: https://github.com/AUTOMATIC1111/stable-diffusion-webui

Expected behavior Have a comparable output with the original float controlnet model from here: https://github.com/Mikubill/sd-webui-controlnet Something like this: 00001-1497947079

Host configuration:

Am I missing something, shall I use another scheduler with different options or is there a better quantized model for controlnet?

bhushan23 commented 5 months ago

nice find @kristoftunner We will have to run this and get back to here with some analysis.