search

Maclory / SPSR

Pytorch implementation of Structure-Preserving Super Resolution with Gradient Guidance (CVPR 2020 & TPAMI 2021)
448 stars 83 forks source link

Cannot reproduce the reported performance using the pretrained checkpoint #33

Closed w238liu closed 3 years ago

w238liu commented 3 years ago

Hi Maclory,

Thanks for your excellent work on SISR and for your generosity to share your code. However, I've met some problems in reproducing the reported results. I am using the pretrained weights in experiments/pretrain_models/spsr.pth, and testing on Set5/Set14/B100/Urban100. I am using the psnr/ssim functions provided in utils\util.py to measure the difference between HR and SR images. I generally find the average PSNR/SSIM on the four databases are much lower than the values reported in the paper. Am I missing something?

I only added a few lines to test.py, and modified the config file test_spsr.json to include HR ground-truth while testing.

test.py:

sr_img = util.tensor2img(visuals['SR']) # uint8 hr_img = util.tensor2img(visuals['HR']) # uint8 psnr = util.calculate_psnr(sr_img, hr_img) ssim = util.calculate_ssim(sr_img, hr_img)

test_spsr.json: { "name": "SPSR", "model": "spsr", "scale": 4, "gpu_ids": [0], "suffix": "_sr",

"datasets": { "test_1": { "name": "set5", "mode": "LRHR", "dataroot_LR": "../../datasets/benchmark/Set5/LR_bicubic/X4", "dataroot_HR": "../../datasets/benchmark/Set5/HR" }, "test_2": { "name": "set14", "mode": "LRHR", "dataroot_LR": "../../datasets/benchmark/Set14/LR_bicubic/X4", "dataroot_HR": "../../datasets/benchmark/Set14/HR" }, "test_3": { "name": "b100", "mode": "LRHR", "dataroot_LR": "../../datasets/benchmark/B100/LR_bicubic/X4", "dataroot_HR": "../../datasets/benchmark/B100/HR" }, "test_4": { "name": "urban100", "mode": "LRHR", "dataroot_LR": "../../datasets/benchmark/Urban100/LR_bicubic/X4", "dataroot_HR": "../../datasets/benchmark/Urban100/HR" } }, "path": { "root": "experiments/pretrain_models", "pretrain_model_G": "experiments/pretrain_models/spsr.pth" },

"network_G": { "which_model_G": "spsr_net", "norm_type": null, "mode": "CNA", "nf": 64, "nb": 23, "in_nc": 3, "out_nc": 3, "gc": 32, "group": 1 } }

Thanks a lot!

Maclory commented 3 years ago

Hi, sorry for the late reply. The reason is that the results reported on the paper are calculated by the code in evaluation toolbox. The functions in util.py are just for validation. There are differences in values because the two kinds of functions are based on different color space.