search

yuanzhi-zhu / DiffPIR

"Denoising Diffusion Models for Plug-and-Play Image Restoration", Yuanzhi Zhu, Kai Zhang, Jingyun Liang, Jiezhang Cao, Bihan Wen, Radu Timofte, Luc Van Gool.
https://yuanzhi-zhu.github.io/DiffPIR/
MIT License
368 stars 31 forks source link

Bad results on SISR #2

Closed pzq-xjtu closed 1 year ago

pzq-xjtu commented 1 year ago

Thank you very much for the work. The generated SR images don't match the LR images when I run main_ddpir_sisr.py. Is there anything else I should be aware of? Can you help me? 00000_x4_k2_LEH

yuanzhi-zhu commented 1 year ago

Hi @pzq-xjtu .

Did you use the parameters ($\eta, \lambda, ...$) the same as in the paper?

pzq-xjtu commented 1 year ago

Thanks for your reply. I used the default parameters in main_ddpir_sisr.py. The sr_mode is set as 'cubic' since it will have a bug when I set sr_mode as 'blur'. The bug 'RuntimeError: "mean_cuda" not implemented for 'ComplexFloat'' is in https://github.com/yuanzhi-zhu/DiffPIR/blob/80d5c151d364ef16a223c509b78a35287c2d8ad6/main_ddpir_sisr.py#L314. The generated SR images look like better than before when I set inIter=5 and gamma=1/10 in https://github.com/yuanzhi-zhu/DiffPIR/blob/80d5c151d364ef16a223c509b78a35287c2d8ad6/main_ddpir_sisr.py#L63-L64. But the generated SR images do not effectively remove noise in the case. 00000_x4_k2_LEH

yuanzhi-zhu commented 1 year ago

Hi @pzq-xjtu , Thank you for your feedback.

the sr_mode 'cubic' is not equal to 'blur' indeed, and you may need to tune the hyperparameters (gammas and inIter) accordingly.

I have updated this repo with a requirements.txt file (along with other minor fixes), could you try it again with 'blur' mode in a new Python environment? Let's hope the bug will disappear!

pzq-xjtu commented 1 year ago

Yes, the bug disappeared in the new environment. And the quality of the SR images is good. Thank you very much for your help.

I have another question. Can you tell me what the difference between the sr_mode 'cubic' and 'blur'? Does the difference lie in the way of solving eq.(12b)? The sr_mode 'cubic' uses the IBP method in eq.(30) of appendix while the 'blur' adopts the closed-form solution in eq.(31), is right?

yuanzhi-zhu commented 1 year ago

that's right.

Indeed, the closed-form solution relies on the blur kernels, which are learned from data (we just adopt it from DPIR);

Additionally, it is worth mentioning that the IBP method in eq.(30) is similar to the update equation in DDNM.

Please do not hesitate if you have any other questions :)

pzq-xjtu commented 1 year ago

Thank you for your answer. I don't have any more questions at the moment.