SeungjunNah / DeepDeblur_release

Deep Multi-scale CNN for Dynamic Scene Deblurring
671 stars 142 forks source link

[Question] Can this method work with unpaired training set? #33

Closed prash030 closed 5 years ago

prash030 commented 5 years ago

Hi, I am trying to apply this method on a training set that does not have a sharp image ground truth pair for every blurred image. The training set only has a bunch of different blur and sharp images. Would the method still work? Any insights and suggestions will be helpful. Thank you.

SeungjunNah commented 5 years ago

Hi @cardyfib,

Let me talk about a fundamental yet important problem of deblurring.

Since it is impossible to capture a blurry and sharp image pair simultaneously, we didn't have many ways to examine the deblurring performance numerically. We had Kohler dataset which used robotic arms to make camera shake blurs, but they were only limited to 4 scenes and 12 types of kernels.

Our first thought was that we need a benchmark dataset that contains realistic dynamic scene blurs and tried to examine our previous video deblurring work.

Then, we moved on to extending the dataset so that we could apply the simplist learning method: supervised learning in this work.

Then, what could be the next? So far, we have tried benchmarking, and supervised learning. One possible step could be towards 'better' supervised settings. We tried to make our method perform better by making a good dataset, and we may work in the same direction by creating better data. Another step could be learning from unsupervised settings since creating dataset requires much time and care. And your case is where unsupervised learning fits more than supervised learning.

You may want to use adversarial models since they have been useful for conditional image generation without GTs. However, some difficulties are expected: 1) Sharper images in similar circumstance as the blurry images tend to be noisier than blurry images due to shorter shutter time and higher ISO. Discriminator may be easily mislead to noisy solution. 2) Even sharp images contain blur if the scene depth varies a lot. (only sharp at salient objects) Note that people may not want to deblur the blurry background.

I hope you will get good ideas.

prash030 commented 5 years ago

Hi @SeungjunNah Thanks a lot for your detailed response. It helps a lot!