search

Fayeben / GenerativeDiffusionPrior

Generative Diffusion Prior for Unified Image Restoration and Enhancement (CVPR2023)
Apache License 2.0
274 stars 30 forks source link

The issue regarding generating abnormal masks. #4

Closed andytoo952 closed 1 year ago

andytoo952 commented 1 year ago

lowlight image:image after gdp: image mask: image Your work is great and very inspiring to me, but I have a question. When I input images with uneven lighting, why does it generate such masks? This leads to directly overwriting the generated images. Is there any way to avoid this? Thank you for your help!

Fayeben commented 1 year ago

Thanks for asking. Have you changed the size of mask? https://github.com/Fayeben/GenerativeDiffusionPrior/blob/2dcd3c6f5b43c3dd20095611530be8ac21b07656/scripts/sample_x0_enhancement_low_light.py#L173

andytoo952 commented 1 year ago

Yes, I have adjusted the two code segments according to the resolution of the corresponding dataset by changing 400, 600 to 720, 1080. It is worth noting that in this dataset, some images may be partially masked by square blocks.

Fayeben commented 1 year ago

Could you share this dataset with me? I want to have a try and find out where is the problem.

andytoo952 commented 1 year ago

Thank you for your patient help. The dataset can be found at https://flyywh.github.io/CVPRW2019LowLight/. I chose 5801-6000 in the train folder as the npz file. I only had time to enhance 4 images, and 5801 and 5803 had this issue.

Fayeben commented 1 year ago

Hi, all! Thanks for raising this point. And I am sorry for the late reply. Due to the domain gap between LOL and DarkFace, the guidance scale we set for LOL seems a little bit larger for DarkFace. You can change this line and decrease the guidance scale to remove the abnormal mask. We set the guidance scale as 4e4 which works well in our experiments. https://github.com/Fayeben/GenerativeDiffusionPrior/blob/116b875db9b4b6c3bcc8adcfc7dbaedb607e1edb/scripts/sample_x0_enhancement_low_light.py#L259 However, it seems DarkFace is still challenging and we also can not effectively enhance this dataset. I am trying to add L_exp to adjust the lightness. If I have some results, I will update them with you. https://github.com/Fayeben/GenerativeDiffusionPrior/blob/116b875db9b4b6c3bcc8adcfc7dbaedb607e1edb/scripts/sample_x0_enhancement_low_light.py#L109

Fayeben commented 1 year ago

Since there are no more questions, I'll mark it closed. If you still have some questions, please feel free to ask. Thanks!