cannot reproduce PSNR=30.19, SSIM=0.9334

[rimchang]

thanks you for great work.

I use below command line. python run_model.py --input_path=./testing_set --output_path=./testing_res --gpu=0 --model=lstm

I try to test on whole GOPRO-test set, but i cannot reproduce PSNR=30.19, SSIM=0.9334. Could you tell me how to reproduce it?plz.

below code is my matlab code and evaluation result.

source='/home/ubuntu/Desktop/workspace/result/result_GOPRO/SRN_deblur/deblur/';
target='/home/ubuntu/Desktop/workspace/result/result_GOPRO/sharp/';
output='/home/ubuntu/Desktop/workspace/result/result_GOPRO/SRN_deblur/matlab_metric.txt';

dir_list = dir(source);
bool_list = not([dir_list.isdir]);
file_list = dir_list(bool_list);

M = containers.Map();
for i = 1:length(file_list)

    file_name = file_list(i).name;
    deblur = imread(strcat(source, file_name));
    sharp = imread(strcat(target, file_name));
    %sharp = imread(strcat(target, strcat(file_name(1:length(file_name)-4), '.jpg')));

    split_name = strsplit(file_name, '_');
    split_name = split_name(1,1:length(split_name)-1);
    name = join(split_name, '_');
    name = char(name);

    result_psnr = psnr(deblur, sharp);
    result_ssim = 0;
    %reuslt_ssim =ssim(deblur, sharp);

    if ~isKey(M, name)
        M(name)=[1, result_psnr, result_ssim];
    else
        metric_list = M(name);
        metric_list(1) = metric_list(1)+1;
        metric_list(2) = metric_list(2)+result_psnr;
        metric_list(3) = metric_list(3)+result_ssim;

        M(name) = metric_list;
    end

    disp(M(name));    

end

total = [0,0,0];
key_set = M.keys;
fid = fopen(output, 'wt');
for i = 1:length(key_set)
    name = key_set{i};

    metric_list = M(name);
    avg_psnr = metric_list(2)/metric_list(1);
    avg_ssim = metric_list(3)/metric_list(1);

    total(1) = total(1) + metric_list(1);
    total(2) = total(2) + metric_list(2);
    total(3) = total(3) + metric_list(3);

    fprintf(fid, '%s Video PSNR : %4.2f, SSIM : %4.2f, Count : %i \n', name, avg_psnr, avg_ssim, metric_list(1));
end

fprintf(fid, 'Total Video PSNR : %4.2f, SSIM : %4.2f, Count : %i \n', total(2)/total(1), total(3)/total(1), total(1));

GOPR0384_11_00 Video PSNR : 30.13, SSIM : 0.00, Count : 100 GOPR0384_11_05 Video PSNR : 28.53, SSIM : 0.00, Count : 100 GOPR0385_11_01 Video PSNR : 28.62, SSIM : 0.00, Count : 100 GOPR0396_11_00 Video PSNR : 30.48, SSIM : 0.00, Count : 100 GOPR0410_11_00 Video PSNR : 28.67, SSIM : 0.00, Count : 134 GOPR0854_11_00 Video PSNR : 26.38, SSIM : 0.00, Count : 100 GOPR0862_11_00 Video PSNR : 24.99, SSIM : 0.00, Count : 77 GOPR0868_11_00 Video PSNR : 26.04, SSIM : 0.00, Count : 100 GOPR0869_11_00 Video PSNR : 27.82, SSIM : 0.00, Count : 100 GOPR0871_11_00 Video PSNR : 26.25, SSIM : 0.00, Count : 100 GOPR0881_11_01 Video PSNR : 27.69, SSIM : 0.00, Count : 100 Total Video PSNR : 27.87, SSIM : 0.00, Count : 1111

[rimchang]

We need to use GOPRO-test-blur dataset instead of GOPRO-test-blur_gamma. when I test on GOPRO-test-blur, I can get simillar result on paper.

GOPR0384_11_00 Video PSNR : 32.20, SSIM : 0.00, Count : 100 GOPR0384_11_05 Video PSNR : 31.78, SSIM : 0.00, Count : 100 GOPR0385_11_01 Video PSNR : 31.70, SSIM : 0.00, Count : 100 GOPR0396_11_00 Video PSNR : 31.79, SSIM : 0.00, Count : 100 GOPR0410_11_00 Video PSNR : 29.53, SSIM : 0.00, Count : 134 GOPR0854_11_00 Video PSNR : 28.20, SSIM : 0.00, Count : 100 GOPR0862_11_00 Video PSNR : 27.30, SSIM : 0.00, Count : 77 GOPR0868_11_00 Video PSNR : 28.63, SSIM : 0.00, Count : 100 GOPR0869_11_00 Video PSNR : 29.47, SSIM : 0.00, Count : 100 GOPR0871_11_00 Video PSNR : 28.62, SSIM : 0.00, Count : 100 GOPR0881_11_01 Video PSNR : 32.51, SSIM : 0.00, Count : 100 Total Video PSNR : 30.20, SSIM : 0.00, Count : 1111