search

sachinpuranik99 / AOD_Net

AOD Net keras implementation
6 stars 7 forks source link

MSE error as printed in your training #4

Closed Kaustubh1Verma closed 4 years ago

Kaustubh1Verma commented 5 years ago

While training in your notebook I saw the error to be something like 0.02. How is that possible for mse loss between two images. Even when training myself, I am getting loss to be in range of 450 something.

sachinpuranik99 commented 5 years ago

Hi,

Prior to running the model in the notebook on a small subset of RESIDE dataset (around 6000 Images), I had trained the model separately on a GPU cluster on the complete RESIDE dataset of 300K images and loaded trained weights.

If you see cell [9], you can see that I am loading the pre-trained weights as follows.

model.load_weights('aod.h5')

If you are indeed loading the pre-trained weights and observing a large MSE loss/low accuracy on RESIDE dataset, please let me know.

Let me know if my explanation is not clear.

Kaustubh1Verma commented 5 years ago

Hi, Thanks for the reply. 1.The RESIDUE dataset I saw on their page has only 13K in standard and 72K in beta. Could you please link the dataset of 300k images. 2.Yes I have also loaded your pretrained weights and I am getting high mse error which converged up to 450. Also is the model parameters in the notebook same which you used for training on GPU cluster?

Kaustubh1Verma commented 5 years ago

Also I am still not clear as to how the mse loss between predicted and ground truth image can be so low.(0.02) And what's the notion of accuracy when its not a classification task?

sachinpuranik99 commented 5 years ago

Hi, Thanks for the reply. 1.The RESIDUE dataset I saw on their page has only 13K in standard and 72K in beta. Could you please link the dataset of 300k images.

OTS 300k images ITS 100k images

2.Yes I have also loaded your pretrained weights and I am getting high mse error which converged up to 450.

Which dataset are you training it on? I am not sure about the overlap between RESIDE beta and RESIDE-V0. If there is no overlap, then high MSE makes sense and you may have to finetune for few epochs by making all the layers trainable by modifying cell [7] model = get_unet(1, trainable = True)

Also is the model parameters in the notebook same which you used for training on GPU cluster?

Its either aod.h5 or aod_net.h5 and both are part of the repo.

sachinpuranik99 commented 5 years ago

Also I am still not clear as to how the mse loss between predicted and ground truth image can be so low.(0.02)

This is just the train loss. Since I am fine tuning it on the trained weights, the model is overfitting which explains the low MSE loss.

And what's the notion of accuracy when it's not a classification task?

You are right that the accuracy is meaningless in the dehazing setting as its not a classification task. Loss, SSIM and PSNR are the correct metrics for evaluation.

I was using accuracy just as an additional metric to guide the training and was not used for checkpointing/early stopping. In this case keras computes the accuracy as in the implementation here

Kaustubh1Verma commented 5 years ago

Thanks for the response. I will try the changes