search

cswry / OSEDiff

[NeurlPS2024] One-Step Effective Diffusion Network for Real-World Image Super-Resolution
Apache License 2.0
210 stars 12 forks source link

About the stability of finetuning vae with LoRA #31

Closed icandle closed 3 weeks ago

icandle commented 2 months ago

Hi author, thanks for the enlightening work! When I try to reimplement OSEDiff, I find fine-tuning VAE and UNet in one stage is hard to converge. Do you have any recommendations/tricks to stablize this process or they should be trained step by step.

cswry commented 2 months ago

Hello, have you tried a version without using KL loss? How does the model perform in training when only using L2 loss and LPIPS loss?

icandle commented 2 months ago

It will be more stable but unable to generating more details.

cswry commented 3 weeks ago

Hello, we have open-sourced the training code, and we hope this can help you!