search

luosiallen / latent-consistency-model

Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference
MIT License
4.36k stars 227 forks source link

issue with [Combine LCM LoRAs with regular SDXL LoRAs] #41

Closed Roveer closed 11 months ago

Roveer commented 11 months ago

Hi! I have followed your blog SDXL in 4 steps with Latent Consistency LoRAs to inference. But when I going to Combine LCM LoRAs with regular SDXL LoRAs, the quality of the images I got is not as high as your blog. Then I copied your code and re-inferenced, but the quality of the images is still very low. And I don't know why. Is it possible that there is an issue with the diffusers version?

luosiallen commented 11 months ago

update to diffusers=0.23.0 first.

Roveer commented 11 months ago

Yes, I'm using version 0.23.0, but the resulting image is still blurry. And the parameters are all the same as yours. But this happens only in the case Combine LCM LoRAs with regular SDXL LoRAs. image_test just like this.

PeiqinSun commented 11 months ago

I also found this problem. Any solution?

ChenFangDart commented 11 months ago

hello, I am having this issue as well. Any help is appreciated!

luosiallen commented 11 months ago

will check later. Have you guys install peft library?

!pip install -q transformers accelerate
!pip install peft
!pip install diffusers
luosiallen commented 11 months ago

make sure this is correct first: https://huggingface.co/docs/diffusers/main/en/tutorials/using_peft_for_inference

patil-suraj commented 11 months ago

Could you please post a code snippet ?

Roveer commented 11 months ago

Hello, I found some problems in this blog, when the function "pipe.load_lora_weights" loads LoRA, the parameter "adapter_name" is not specified as "lora". The original code: pipe.load_lora_weights(lcm_lora_id) Just code it to: pipe.load_lora_weights(lcm_lora_id, adapter_name="lora") and then you got a satisfying image.

luosiallen commented 11 months ago

Thanks @Roveer ! Will update the blog.