search

Fayeben / GenerativeDiffusionPrior

Generative Diffusion Prior for Unified Image Restoration and Enhancement (CVPR2023)
Apache License 2.0
277 stars 30 forks source link

Mismatch between the code and the paper (low-light enhancement and HDR) #17

Closed opherbambi closed 1 year ago

opherbambi commented 1 year ago

Hi, When I got into the code I noticed that in the running scripts of "sample_x0_enhancement_HDR\low_light.py" in the function "light_cond_fn", the loss function is only the MSE loss, while the calculation of the loss expression with L_exp, L_color, and L_TV are being commented.

image

While in the paper, in Algorithm 2 GDP-x0 those losses do appear (as the Q).

image

Can you please explain it, I probably missed something in the code or I did not understand the paper well.

In addition, why the light factor and mask are updated according to the MSE loss only?

Fayeben commented 1 year ago

Hi, thanks for raising this valuable point. As you can see in the caption of the algorithm, the other three losses are optional. In fact, we can enhance the low-light images only with MSE loss. The other three losses are used to make our method more flexible, for example, L_exp can be used to control the brightness of the restored images. Moreover, if you use these losses together, you can update the light factor and mask via the total loss. If you have more questions, please feel free to ask. Thanks!

opherbambi commented 1 year ago

Thanks When I set the loss to the Reconstruction loss (mse) and the Quality loss (L_exp, L_color, and L_TV) as specified in the Appendix, and update the light factor and mask via the total loss I got black image. Did it happen to you too?

Fayeben commented 1 year ago

Hi, you can change the guidance scale to 5e4 in this line, since we use 2 as the number of optimization per time step. https://github.com/Fayeben/GenerativeDiffusionPrior/blob/116b875db9b4b6c3bcc8adcfc7dbaedb607e1edb/scripts/sample_x0_enhancement_low_light.py#L259C37-L259C37 Moreover, you can change this line to generate images with different lightness. https://github.com/Fayeben/GenerativeDiffusionPrior/blob/116b875db9b4b6c3bcc8adcfc7dbaedb607e1edb/scripts/sample_x0_enhancement_low_light.py#L73

Fayeben commented 1 year ago

Since there are no more questions, I'll mark it closed. If you still have some questions, please feel free to ask. Thanks!