search

greatlog / DAN

This is an official implementation of Unfolding the Alternating Optimization for Blind Super Resolution
231 stars 41 forks source link

Local test the performance of pretrained model, different from the reported value #22

Closed 1LOVESJohnny closed 3 years ago

1LOVESJohnny commented 3 years ago

Hi, thanks for providing detailed codes and it's a great job!

I'm testing the pretrained model in local on self generated Gaussian 8 testset with kernel size = 21x21. And I got the performance as below.

Method | Scale | Set5 | Set5 | Set14 | Set14 | B100 | B100 | Urban100 | Urban100 | Mangan109 | Manga109 -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | --   |   | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM DANv1 local test | x4 | 31.75 | 0.8833 | 28.19 | 0.7767 | 27.42 | 0.7214 | 25.77 | 0.7683 | 30.39 | 0.9021 DANv1 | x4 | 31.89 | 0.8864 | 28.42 | 0.7687 | 27.51 | 0.7248 | 25.86 | 0.7721 | 30.50 | 0.9037

Though the local test perf is also excellent, it's different from the reported one (quote from paper). I'm a little bit confused... Not sure whether there is a mistake made during testing... Expect for your reply!

greatlog commented 3 years ago

How do you generate the testing datasets? If they are generated with different codes, the results may be different, especially the setting of bicubic downsampling.

1LOVESJohnny commented 3 years ago

How do you generate the testing datasets? If they are generated with different codes, the results may be different, especially the setting of bicubic downsampling.

Hi, I generated the test set according to the code and reproduced in local. May I ask if you can share your testset?

Thanks!

greatlog commented 3 years ago

The whole test set or visual results of Setting1 are too large to be shared. I upload the test set of Set5 here. Maybe you can check your generating code by comparing my set with your generated set. x4Set5.zip

1LOVESJohnny commented 3 years ago

The whole test set or visual results of Setting1 are too large to be shared. I upload the test set of Set5 here. Maybe you can check your generating code by comparing my set with your generated set. x4Set5.zip

Thanks for your sharing. I'll double check in local. Will update the status soon.

1LOVESJohnny commented 3 years ago

Latest update.

I've tested with the shared testset and the perf matches with the reported value. Seems like there should be a bug in my test data generation module.

Thanks for your help @greatlog.

Young-Zzz commented 3 years ago

Latest update.

I've tested with the shared testset and the perf matches with the reported value. Seems like there should be a bug in my test data generation module.

Thanks for your help @greatlog.

Hi,I have committed a new opend issue,can you figure out where is the problem? My result is much lower than paper and I still can't fix it.