search

TaoHuang2018 / Neighbor2Neighbor

Neighbor2Neighbor: Self-Supervised Denoising from Single Noisy Images
BSD 3-Clause "New" or "Revised" License
248 stars 39 forks source link

The Problem about Generating Training Dataset #11

Open polyuLi opened 2 years ago

polyuLi commented 2 years ago

I have a question about the training dataset. Why can you assume we can obtain multiple degradation versions of the same latent images in unsupervised denoising? The Line 370 in the "Train.py" script, you add different noise samples on clear images in each epoch, which results in generating multiple degradation versions of the same latent images. I believe this operation violates the statement in your paper. "The proposed self-supervised framework aims at training denoising networks with only single noisy images available" (Not multiple noisy images as N2N).

That is a very crucial problem. If we can generate multiples noisy versions of the same latent image, for example, 100 versions as you do (100 epochs), we can simply average all these 100 counterparts to generate the pseudo ground truth and use them for supervision learning. In fact, collecting multiple noisy versions of the same latent images and averaging all of them to generate the pseudo ground truth is the way for real-world datasets, e.g., DND and SIDD.

TaoHuang2018 commented 2 years ago

This is a good question. Here I would like to give the following explanations:

  1. The aim of this synthetic experiment is to evaluate the performances of single images denoising methods. We refer to the settings of High-Quality Self-Supervised Deep Image Denoising (in https://github.com/NVlabs/selfsupervised-denoising), including noisy image generator and the training process.
  2. As we do not utilize the correlations between different noisy images per ground-true image, i.e., we do not use paired clean images or paired noisy images to calculate the loss function and update the network. Thus, we do not violate the assumption of single noisy images available when training with our algorithm.
  3. In fact, we view the 100 versions of noisy images as separate input images. Besides, as the pytorch training dataloader is shuffled, we do not know which noisy images has the same latent clean image. Therefore, it would be unavailable to average over 100 counterparts to generate the pseudo clean image.