search

sefibk / KernelGAN

Other
337 stars 77 forks source link

why not directly save the params of Generator for downscaling? why not non-linear? #65

Open hitsz-zuoqi opened 3 years ago

hitsz-zuoqi commented 3 years ago

As shown in paper, the generator is equaled to a simple linear kernel, then why we still need to use 'imresize' func to resize the picture, can we save all weights of generator for downscaling?

Another question, I see the post_process_k func and I think even a non-linear generator can be postprocessed as a fine filter, and also the non-linear filter can be computed by the same way.

Could you please explain about the two questions?

sefibk commented 3 years ago

As shown in paper, the generator is equaled to a simple linear kernel, then why we still need to use 'imresize' func to resize the picture, can we save all weights of generator for downscaling?

Yes. But that will be computationally more expensive. Even simply the first layer converting to 64 channels is much more expensive

Another question, I see the post_process_k func and I think even a non-linear generator can be postprocessed as a fine filter, and also the non-linear filter can be computed by the same way.

A non-linear filter would be able to "overfit" and produce random patches of the image regardless of the input. That way we would not be learning a downscaling function. Additionally, what does a non-linear kernel relate to in real-life?

hitsz-zuoqi commented 3 years ago

I see the multiplicative noise experiments in paper, maybe the kernel after multiplication is non-linear? In real-life, non-linearity can also be caused by image compression or other factors?

sefibk commented 3 years ago

Definitely can, the problem is I am not sure of how to enforce the Generator to learn only "realistic" models. As I mentioned - it can easily "memorize" the patches of the image and output only those patches and fool the discriminator. There is actually a great work that does exactly that (InGAN)