search

justinpinkney / stable-diffusion

MIT License
1.44k stars 261 forks source link

Converting trained stable diffusion to diffusers reduces image quality #44

Open nikitalokhmachev-ai opened 1 year ago

nikitalokhmachev-ai commented 1 year ago

Hello, thanks for the tutorial, it's been extremely helpful! I've trained my own model on the lambda gpu according to the tutorial and converted the model to diffusers to test the model in colab. However, for some reason, when I load the model, the quality seems to be significantly worse. I tried converting the model with both --use_ema and no --use_ema. --use_ema improved the quality slightly but nothing significant (I might also be biased that the quality improved in the first place). Do you know what the problem might be?

When performing the conversion, do I need to use the yaml config file I used during the training or the original stable diffusion one? (I was using the one I created)

065294847 commented 1 year ago

Did you find a solution to this?

nikitalokhmachev-ai commented 1 year ago

Unfortunately, I did not. However, I believe a good solution would be either fine-tuning a diffusers model in dreambooth with textual inversion. If you have any other ideas, feel free to share.

justinpinkney commented 1 year ago

The yaml config file shouldn't matter. And the most obvious difference I found before was using ema vs not. I know diffusers has undergone a few new versions since I released this, so maybe try the script in their repo instead? https://github.com/huggingface/diffusers/blob/main/scripts/convert_original_stable_diffusion_to_diffusers.py