search

KuldeepKulkarni / ReconNet

ReconNet: CVPR 2016
https://kuldeepkulkarni.github.io/reconnet.htm
MIT License
44 stars 25 forks source link

Reproduction of the results #3

Open gxstudy opened 6 years ago

gxstudy commented 6 years ago

Dear Kuldeep,

I have trouble reproduce the results, if using mr=0.25, the code provided caffemodel achieves the following results:

Measurement Rate = 0.25 Cleared 0 solvers and 0 stand-alone nets Monarch.tif: PSNR = 24.317236 dB, Time = 0.017663 seconds Parrots.tif: PSNR = 25.593014 dB, Time = 0.016800 seconds barbara.tif: PSNR = 23.250978 dB, Time = 0.011068 seconds boats.tif: PSNR = 27.303005 dB, Time = 0.011923 seconds cameraman.tif: PSNR = 23.151989 dB, Time = 0.028505 seconds fingerprint.png: PSNR = 25.574577 dB, Time = 0.035927 seconds flinstones.png: PSNR = 22.452606 dB, Time = 0.028761 seconds foreman.tif: PSNR = 29.473907 dB, Time = 0.009909 seconds house.tif: PSNR = 28.461853 dB, Time = 0.013917 seconds lena256.tif: PSNR = 26.544273 dB, Time = 0.014411 seconds peppers256.png: PSNR = 24.766825 dB, Time = 0.023207 seconds

But if I train by myself, where all the parameters are kept the same, basically, I loaded ../phi/phi_0_25_1089.mat when generating the train.h5 and test.h5, then made std= 0.05 in ReconNet_arch.prototxt as suggested. Then use the iter=1000000 caffemodel(reconnet_0_25_retrain_iter_1000000.caffemodel) for testing. The results are:

Measurement Rate = 0.25 Cleared 0 solvers and 0 stand-alone nets Monarch.tif: PSNR = 22.158499 dB, Time = 0.018842 seconds Parrots.tif: PSNR = 23.664309 dB, Time = 0.010023 seconds barbara.tif: PSNR = 22.776989 dB, Time = 0.008880 seconds boats.tif: PSNR = 25.165928 dB, Time = 0.008666 seconds cameraman.tif: PSNR = 21.711927 dB, Time = 0.008943 seconds fingerprint.png: PSNR = 22.167679 dB, Time = 0.042384 seconds flinstones.png: PSNR = 19.717672 dB, Time = 0.025763 seconds foreman.tif: PSNR = 28.166494 dB, Time = 0.028865 seconds house.tif: PSNR = 26.962069 dB, Time = 0.009218 seconds lena256.tif: PSNR = 24.913033 dB, Time = 0.026679 seconds peppers256.png: PSNR = 23.098946 dB, Time = 0.007586 seconds

which are much lower than the provided model. Could you please help me check whether I miss some setup, thanks a lot.

Best, Cindy

KuldeepKulkarni commented 6 years ago

Hi Cindy,

Thanks for your message. Do you have the training log?

-Kuldeep

gxstudy commented 6 years ago

thanks a lot, attached is the log. I made it save a model every 10000 iterations, otherwise there gonna be too many output models. reconnet_0_25_xin_retrain.log

KuldeepKulkarni commented 6 years ago

Thanks. If I remember correctly, the final 'Test net output #0: loss' was 0.52 when the training saturated and this happened earlier than 10^6 iterations. But in your case, it is 0.85. I'll try to find my log file for comparison. Can you repeat the training from scratch, and see if there is any change? But why do you want to train again when you have the model for 0.25 measurement rate?

gxstudy commented 6 years ago

HI, Kuldeep,

I retrained last night with completely same parameters, but it seems the results got even worst.

Measurement Rate = 0.25 Cleared 0 solvers and 0 stand-alone nets Monarch.tif: PSNR = 21.822254 dB, Time = 0.025000 seconds Parrots.tif: PSNR = 23.397217 dB, Time = 0.008815 seconds barbara.tif: PSNR = 22.695799 dB, Time = 0.008789 seconds boats.tif: PSNR = 24.679037 dB, Time = 0.008580 seconds cameraman.tif: PSNR = 21.271360 dB, Time = 0.009084 seconds fingerprint.png: PSNR = 21.138027 dB, Time = 0.023954 seconds flinstones.png: PSNR = 19.161001 dB, Time = 0.023919 seconds foreman.tif: PSNR = 27.706085 dB, Time = 0.009677 seconds house.tif: PSNR = 26.314543 dB, Time = 0.008981 seconds lena256.tif: PSNR = 24.522911 dB, Time = 0.008081 seconds peppers256.png: PSNR = 22.813314 dB, Time = 0.007831 seconds

Attached is the log file. Thanks a lot for the help. reconnet_0_25_xin_retrain2.log

KuldeepKulkarni commented 6 years ago

Did you initialize the first layer (the FC layer) with phi^T ?

gxstudy commented 6 years ago

Hi, Kuldeep,

I didn't see any place you intend to initialize the first layer with phi^T, I think in ReconNet_arch.prototxt, you initialize the weights by Gaussian with std=0.05. And there is no pre-loaded parameters in train.sh. Please let me know if my understanding is wrong. Thanks a lot for the help.

KuldeepKulkarni commented 6 years ago

So, you cannot initialize the first layer with phi^T by editing ReconNet_arch.prototxt. What you can do is the following. 1) Train for say, 1 iteration, using train.sh 2) Save the model, and load its weights using matcaffe or pycaffe, change the first layer to phi^T and save the model again and call it ReconNet_0_25_phi.caffemodel. 3) Change the second line in train.sh to '../../../build/tools/caffe train --solver ReconNet_solver.prototxt -weights ReconNet_0_25_phi.caffemodel', and run it.

gxstudy commented 6 years ago

Hi, Kuldeep,

Thanks a lot, but I have never changed a model parameter, could you please share with us the ReconNet_0_25_phi.caffemodel or the matlab code or python code to generate it? The code will be better since we want to try mr as 0.01 and 0.1 later, that would be a big help. Thank you very much.

TaihuLight commented 5 years ago

@KuldeepKulkarni @gxstudy ReconNet is a classic CS algorithm, could you provide the trained models using MR=0.40 and MR=0.50, since it is hard to train model successfully using MR=0.40 and MR=0.50.

y0umu commented 5 years ago

Did you initialize the first layer (the FC layer) with phi^T ?

Hi KuldeepKulkarni, Sorry to disturb but I really don't see the point why you want to initialize the first layer with phi^T. Weights of the first layer will finally be updated by gradient descend and the initialized phi^T will be destroyed. Your paper does not seem to touch on this either. Could you please kindly provide some clues on your motivation? Or you simply found that initializing with phi^T is better than random gaussian through laborious trainning?