Negative PSNR value occured in training phase

[zinuoli]

Hi, it is a great and constructive work and I really appreciate you can release your code. May I kindly ask if you have encountered the similar problem like the PSNR became negative in the middle of training phase and there will not be better model saved in the next training period? I am using the same dataset as yours and the code is not modified. Thank you so much if you could give me a tip and wish you have a nice day :)

[ZhexinLiang]

Hi zinuoli, thanks for your interest in our work.

Sorry for confusing you. It means the initialization is finished and turn to the fine-tuning stage. In initialization stage, I use PSNR between the input and the output to determine whether the reconstruction results are good or not. In fine-tuning stage, I use the negative value of the loss to decide whether to save the enhanced results or not (The larger negative loss, the better the enhanced results.). Therefore, the value '213.401' in your screenshot represents the current loss for this result, not the PSNR.

I may update this expression in the code at a later time.

If you have any other question, please feel free to ask. :)

[zinuoli]

Thank you for your help :) I do have another question and want to discuss with you.

I read your code, in the initialization stage you calculate the PSNR to determine the inference quality. However, in trainloader we actually only load unpaired data, how can we compute PSNR in these images?

I may lost something when I was reading your code...So pls forgive me if I say something wrong, this is just a friendly talk and not offensive.

Thank you :)

[zinuoli]

Oh! Btw, in the fine tuning stage I didn't notice any further results saved, it this normal?

[ZhexinLiang]

Thank you for your help :) I do have another question and want to discuss with you.

I read your code, in the initialization stage you calculate the PSNR to determine the inference quality. However, in trainloader we actually only load unpaired data, how can we compute PSNR in these images?

I may lost something when I was reading your code...So pls forgive me if I say something wrong, this is just a friendly talk and not offensive.

Thank you :)

Yes, you are right. In the fine-tuning stage there is no paired data. So we do not calculate the PSNR but use the loss instead to determine the inference quality, just as I explained above.

[zinuoli]

Thank you so much for your reply, may I ask in the fine tuning stage I didn't notice any further results saved, it this normal? Or maybe I encountered something wrong?

[ZhexinLiang]

I guess you are trying to train from scratch. I also found this phenomenon when training from randomly initialized models.

You may first check if the reconstruction results are right. If not, which should not happend if you use the lateset version of my code, you should rerun the reconstruction stage.

If the reconstruction is right, this may be caused by the initialization strategy making the enhancement model hard to find the right way to fit at first. Based on my experiments, this problem tends to be resolved if you train the model for enough iterations (I forget the exact number of the training iterations to resolve this, may be after 25k iterations). You can also use the pretrained enhancement model checkpoint to resolve this.

Feel free to have more discussions if you want. :)

[zinuoli]

Many thanks for your patient reply :) I am need training from scratch, and I will try more in the near future.

Have a good day!

[zinuoli]

Hi, sorry to bother you again, may I ask what is this code for? https://github.com/ZhexinLiang/CLIP-LIT/blob/e644163311bc833da93a9e74ee8961c21711cb3b/train.py#L249C28-L249C28

I notice your paper mentioned identity loss to constrain the image content, and here you compare the enhanced image with the original image, are they for the same purpose?

[ZhexinLiang]

Hi Zinuo, this is for the reconstruction stage to use PSNR to determine whether to save the new reconsturction results or not.

[zinuoli]

Hi, thanks for your reply. But the PSNR is calculated based on the low light images and enhanced images, not target images and enhanced images. I feel a bit confused here.

[ZhexinLiang]

This is the very first stage for initializing the enhancement model. At the beginning, we only do the reconstruction using the identity loss. During reconstruction, we only have low-light images and reconstructed results. We calculate the PSNR to determine if the reconstrution is complete.

This reconstruction process is not emphasized in the paper.

After this, we incorporate CLIP-enhance loss into the training process to obtain enhanced results, which is the initial enhancement network training in stage (a) in figure 4 in the paper. And enhanced images are not paired with target images, so we cannot calculate PSNR between target and enhanced results.

[ZhexinLiang]

You can also check section 3.1 in the paper. The code you mentioned is for the stage noted in yellow.

[ZhexinLiang]

Hi, sorry to bother you again, may I ask what is this code for? https://github.com/ZhexinLiang/CLIP-LIT/blob/e644163311bc833da93a9e74ee8961c21711cb3b/train.py#L249C28-L249C28

I notice your paper mentioned identity loss to constrain the image content, and here you compare the enhanced image with the original image, are they for the same purpose?

The identity loss is to constrain the training process. The PSNR (the comparison between enhanced image and the original image) is only to determine whether to save the results or not, that is, this PSNR won't influence the training process. You can even delete this code if you don't want to save the reconstruction results.

[zinuoli]

Thank you for your patient reply :) Now I see the purpose, wish you all the best.