sefibk / KernelGAN

Other
337 stars 77 forks source link

the code is wrong theoritically, please check #47

Open HLzju opened 3 years ago

HLzju commented 3 years ago

I think 'depthwise conv' is preferred in function 'resize_tensor_w_kernel' of 'util.py' because we usually apply downsampling process to R,G,B seperatly.

Anyway ,the author makes the bicubic loss shutdown after hundreds of iterations, so this bug goes away.

sefibk commented 3 years ago

I don't understand why you think the code is wrong. The downscaling is done identically to all 3 channels. Anyway, you are welcome to test this and open a PR if performance is maintained on DIV2K-RK

HLzju commented 3 years ago

I got a grayscale image(even though it has 3 channels) when I debug the ouput tensor of 'resize_tensor_w_kernel' in python file 'util.py', I've read the paper and I think it should be a colored image.

I tried to change the 'resize_tensor_w_kernel' as below:

def resize_tensor_w_kernel(im_t, k, sf=None): """Convolves a tensor with a given bicubic kernel according to scale factor"""

Expand dimensions to fit convolution: [out_channels, 1, k_height, k_width]

k = k.expand(im_t.shape[1], **1**, k.shape[0], k.shape[1])
# Calculate padding
padding = (k.shape[-1] - 1) // 2
return F.conv2d(im_t, k, stride=round(1 / sf), padding=padding,**groups=im_t.shape[1]**)#

I got a colored image.

maybe, there's something I missed about the paper,I would appreciate it if you help me with my doubts.

sefibk commented 3 years ago

You might be correct about the gray scale - AFAIK it is not supposed to be gray. I will try to look at it but it won't be in the near future - I am very busy on other issues

fedral commented 3 years ago

I got a grayscale image(even though it has 3 channels) when I debug the ouput tensor of 'resize_tensor_w_kernel' in python file 'util.py', I've read the paper and I think it should be a colored image.

I tried to change the 'resize_tensor_w_kernel' as below:

def resize_tensor_w_kernel(im_t, k, sf=None): """Convolves a tensor with a given bicubic kernel according to scale factor"""

Expand dimensions to fit convolution: [out_channels, 1, k_height, k_width]

k = k.expand(im_t.shape[1], 1, k.shape[0], k.shape[1])

Calculate padding

padding = (k.shape[-1] - 1) // 2 return F.conv2d(im_t, k, stride=round(1 / sf), padding=padding,groups=im_t.shape[1])#

I got a colored image.

maybe, there's something I missed about the paper,I would appreciate it if you help me with my doubts.

I have came across the same bug here, and I solved it in the same way. This bug has already been pointed out in a closed iussue. if groupwise convolution is not used, it will generate black image, normal blured image with "groupwise" parameter.

By the way, in recent CVPR 2020 super resolution competition, "Real-World Super-Resolution via Kernel Estimation and Noise Injection" used your work kernelGAN to gain the first place in real-world SR. This might be the reason why more and more people are checking and trying to reproduce your results of KernelGAN.

fedral commented 3 years ago

I got a grayscale image(even though it has 3 channels) when I debug the ouput tensor of 'resize_tensor_w_kernel' in python file 'util.py', I've read the paper and I think it should be a colored image.

I tried to change the 'resize_tensor_w_kernel' as below:

def resize_tensor_w_kernel(im_t, k, sf=None): """Convolves a tensor with a given bicubic kernel according to scale factor"""

Expand dimensions to fit convolution: [out_channels, 1, k_height, k_width]

k = k.expand(im_t.shape[1], 1, k.shape[0], k.shape[1])

Calculate padding

padding = (k.shape[-1] - 1) // 2 return F.conv2d(im_t, k, stride=round(1 / sf), padding=padding,groups=im_t.shape[1])#

I got a colored image.

maybe, there's something I missed about the paper,I would appreciate it if you help me with my doubts.

here is the link https://github.com/sefibk/KernelGAN/issues/28

sefibk commented 3 years ago

Could you please open a PR and I will go over it?