ZhexinLiang / CLIP-LIT

[ICCV 2023, Oral] Iterative Prompt Learning for Unsupervised Backlit Image Enhancement
https://zhexinliang.github.io/CLIP_LIT_page/
269 stars 23 forks source link

about the CLIP_enhance loss #16

Closed diadestiny closed 8 months ago

diadestiny commented 8 months ago

Hello, in your paper, why is the CLIP_enhance loss using the negative text prompt T_n instead of the positive text prompt T_p? Shouldn't the enhance result of the model be more closely matched with the positive text prompt? image If I want to use this loss for other image restoration tasks, should I replace T_n in this loss with T_p?

ZhexinLiang commented 8 months ago

Hi @diadestiny, the loss function $L_{clip}$ in our paper is correct and should not be changed.

The loss function $L_{clip}$ is a softmax function, which means the $T_p$ is already included in the consideration.

And for a loss function, during training, the value will continue to decrease untill converge. The lower the $L_{clip}$ , the results will be more away from the $T_n$ ,that is, more closer to the $T_p$ .

If you replace $T_n$ with $Tp$ , the value of the new loss will equal to $1-L{clip}$ , and if this loss decreases during training, the results will be away from the $T_p$ , which is not what we want.

If you still want to replace $T_n$ with $T_p$ , please use the opposite number of the replaced loss. But the effect will be the same as the loss in our paper.