search

yuanzhi-zhu / DiffPIR

"Denoising Diffusion Models for Plug-and-Play Image Restoration", Yuanzhi Zhu, Kai Zhang, Jingyun Liang, Jiezhang Cao, Bihan Wen, Radu Timofte, Luc Van Gool.
https://yuanzhi-zhu.github.io/DiffPIR/
MIT License
388 stars 31 forks source link

Some questions about the implemented equations in the codes #23

Closed dawnlh closed 11 months ago

dawnlh commented 11 months ago

Thanks for sharing the great work! I can run the code successfully， but I'm confused with some implementations in main_ddpir_deblur.py and respectfully request your help.

https://github.com/yuanzhi-zhu/DiffPIR/blob/88b9b81568c6d0fd84d8ad2092ef856be6afa10d/main_ddpir_deblur.py#L91-L96

In this part, the variables noise_model_t and t_start are overrided. The initial setting seems to determine the start point for the reverse diffusion (instead of directly from the random noise), but why override them. (I tried the original setting and the performance got worse)

https://github.com/yuanzhi-zhu/DiffPIR/blob/88b9b81568c6d0fd84d8ad2092ef856be6afa10d/main_ddpir_deblur.py#L342-L347

These lines are corresponding to Eq.14-15 in the paper, i.e. image

  1. I didn't figure out which equation eta_sigma in Line.345 corresponses to in the paper. It looks like $\sigma_t$ in Eq.12 (for DDPM case) of the DDIM paper, but not the same.

  2. In Line 346-347, the equation seems to be a combination of Eq.14 and Eq.15, i.e., it contains both $\sigma_{\etat}$ and $\zeta$. I know $\eta$ is set to 0 (thus $\sigma{\eta_t}=0$, this line corresponse to Eq.15 ) in this code, but I'm confused about the equation's original meaning.

yuanzhi-zhu commented 11 months ago

Hi @dawnlh , thanks for your interest in our work.

  1. For noise_model_t, we use it to skip solving the analytic solution at the end of the sampling process. The intuition behind this is that at a high signal-noise ratio region, the diffusion sampling process does nothing but iteratively denoising, and the measurement information $y$ may not be necessary.
  2. For t_start, we use it to shortcut to a closer initial point for faster sampling. You may find why on page 18 of this slide.
  3. Sorry for the confusion and you are mostly correct that In Line 346-347, the equation seems to be a combination of Eq.14 and Eq.15, where eta_sigma is the $\sigma_{\eta_t}$ in eq (14) (the DDIM formula). The $\zeta$ in eq(15) is a bit hand-wavy and more like an experimental result.

We intend to leave all the tunable hyperparameters that appear in the paper in this code base for people to try. By tuning them, I believe one can better understand this method.

Hope these can address your concerns and please do not hesitate if you have any other questions

dawnlh commented 11 months ago

Got it! Thank you for your prompt response and kind explanation. 😄