search

omerbt / MultiDiffusion

Official Pytorch Implementation for "MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generation" presenting "MultiDiffusion" (ICML 2023)
https://multidiffusion.github.io/
997 stars 59 forks source link

Why is averaging over overlapping regions necessary? #21

Open yxchng opened 1 year ago

yxchng commented 1 year ago

Why can't we just do non-overlapping generation without averaging? I presume you have tried and found that averaging gives better results. Is there any intuition or theory for that?

bishalsahooviga commented 7 months ago

Can you suggest ways to to average and overlap masks? I went through the paper and the codes but I think I missed some important part.

stanifrolov commented 3 weeks ago

Hello @yxchng, this is a great question that I had as well. The averaging between overlapping regions is necessary to create a coherent and smooth transition between the individual denoised patches. However, it turns out that you can also simply shift non-overlapping denoising windows through the timesteps such that borders in one timestep are corrected in the next which results in faster image generation, see our analysis here https://spotdiffusion.github.io/