Open hcwang95 opened 4 years ago
I found that there may be some bugs in the latest version of the code. I tried to switch the code to the previous version(2e1a266832ab3a2dd855c98c34387c47a4ebec01) and got a PSNR value similar to that in the paper.
Thanks a lot, @hanlinwu! I will try it soon and update here if it works well.
Here is the update. By rolling back to the commit as @hanlinwu pointed out, I successfully achieved around 41dB PSNR on the Set5 validation set. However, when I test the pre-trained model on B100 (which I created myself by Octave rather than Matlab), the PSNR is even lower (around 27dB).
I found that I could not actually reproduce the input image by running the dataset generation script in MATLAB using Octave. @XuecaiHu Would you mind if you could share the B100 dataset that you use for the training or sharing the Matlab version that you run? Thanks a lot!
Did you solve this problem?@hcwang95
@reddandelion217 No, I've tried one another training dataset and still cannot reproduce the performance in the paper. Not sure where is the problem.
@hcwang95 After analyzing the file history, I think the problem comes from the code below in file trainer.py, because there is no reason to do this: i = 1 h, w,_ = pos_mat.size() while(pos_mat[i][0][0]<= 1e-6 and i<h): i = i+1
j = 1
#pdb.set_trace()
h, w,_ = pos_mat.size()
while(pos_mat[0][j][1]<= 1e-6 and j<w):
j = j+1
pos_mat_small = pos_mat[0:i,0:j,:]
and the code below in file metardn.py: local_weight = self.repeat_weight(local_weight,scale_int,x.size(2),x.size(3))
@hcwang95 Hello, have you solved the reproduction problem?
I have a similar problem with you. I used the script ./generate_LR_metasr_X1_X4_idealboy.m to prepare the B100 dataset. Then I used the downloaded model_1000.pt to evaluate the PSNR and SSIM results.
What I got at scales 1.1 ~ 2.0 is as follows: |
X1.1 | X1.2 | X1.3 | X1.4 | X1.5 | X1.6 | X1.7 | X1.8 | X1.9 | X2.0 | |
---|---|---|---|---|---|---|---|---|---|---|---|
PSNR | 27.71 | 27.73 | 27.69 | 27.26 | 27.98 | 27.23 | 27.10 | 27.02 | 26.97 | 28.33 | |
SSIM | 0.8412 | 0.8417 | 0.8380 | 0.8223 | 0.8424 | 0.8147 | 0.8086 | 0.8030 | 0.7975 | 0.8398 |
The PSNR results I got are much lower than that from the paper. @XuecaiHu Could you please help solve the problem? Thank you!
@hcwang95 Can you provide some details about how you solve the low-PSNR problem? I meet the same problem with you. I feel confused about @hanlinwu 's comment -- on the preview version
@supercaoO can you check the output of the h_project_coord and w_project_coord?
and show me the results @supercaoO
@XuecaiHu Thanks for your reply. I have emailed you (xuecai.hu@cripac.ia.ac.cn) the results.
@liangheng96 Hello. After cloning the repository, maybe you can try git reset --hard 2e1a266832ab3a2dd855c98c34387c47a4ebec01
to switch it to the previous version of the repository. I think that is what @hanlinwu means.
@supercaoO Thank you! I know what @hanlinwu means by now. Now I swith to the previous version, and then I can get the similar results to the paper. Thank you very much!
I think @reddandelion217 's comment is right. I delete these codes and then get the right PSNR.
@hcwang95 After analyzing the file history, I think the problem comes from the code below in file trainer.py, because there is no reason to do this: i = 1 h, w,_ = pos_mat.size() while(pos_mat[i][0][0]<= 1e-6 and i<h): i = i+1
j = 1 #pdb.set_trace() h, w,_ = pos_mat.size() while(pos_mat[0][j][1]<= 1e-6 and j<w): j = j+1 pos_mat_small = pos_mat[0:i,0:j,:]
and the code below in file metardn.py: local_weight = self.repeat_weight(local_weight,scale_int,x.size(2),x.size(3))
Hi, do you see any errors in meta.py, saying 'ValueError: only one element tensors can be converted to Python scalars'? Thanks!@hcwang95
Any updates on this?
Thanks for providing the detailed code with instructions to train and test.
I am curious if I have any problem with having pretty low PSNR during the evaluation of my trained model and even the pre-trained model downloaded from Google Drive.
The trained model got trained followed by the instruction in README but the PSNR for 1.1x on B100 is only around 29dB. Then I evaluated the pre-trained model and PSNR for 1.1x is also around 29dB.
Would you mind telling me whether the pre-trained model is the model generating the scores in the paper or preliminary model? Thanks a lot!