search

LingxiaoYang2023 / DSG2024

Official pytorch repository for “Guidance with Spherical Gaussian Constraint for Conditional Diffusion”
43 stars 2 forks source link

Using SD1.4 cannot reconstruct image y when applying L(\hat{x0}, y) #9

Closed Sutongtong233 closed 3 weeks ago

Sutongtong233 commented 1 month ago

Hi, when I use code in SD_style with SDv1.4, and I want to reconstruct a image: 

def get get_residual(x0, y):
  differences = x0 - y
  return differences

The reconstruction is unsatisfied. input:image output:image Does it because the limitation of SD model (VAE limitation)?

LingxiaoYang2023 commented 4 weeks ago

This could be caused by several reasons: 1. The error in the VAE as you said. 2. The data domain of Stable Diffusion is too large, leading to the need for more NFEs for optimization. 3. The generated image quality seems low; you might need to use a smaller guidance rate and a larger interval.