Artifacts when denoising AWGN

[m-tassano]

Hi! I'm getting loads of artifacts when denoising sequences with AWGN (see a couple of examples of sequences from the DAVIS dataset below) Is this normal? Thanks

[clausmichele]

I am currently working on a new version of the code with the training part, since I guess there are some problems with tensorflow and the stored weights. As soon as I finish I will release it here.

[clausmichele]

Hi! I'm getting loads of artifacts when denoising sequences with AWGN (see a couple of examples of sequences from the DAVIS dataset below) Is this normal? Thanks

Could you please share with me the data you tried to denoise? How did you add AWGN? I will use it for testing..

[m-tassano]

I used different sources, for example the DAVIS testset https://data.vision.ee.ethz.ch/csergi/share/davis/DAVIS-2017-test-dev-480p.zip and some sequences from the Derf dataset downscaled to 960x540 (e.g. crowd_run ftp://vqeg.its.bldrdoc.gov/HDTV/SVT_MultiFormat/ )

For the AWGN, I modified the definition of the noisy, noisy1, noisy2 vars like the following

noisy = test + np.random.normal(loc=.0, scale=(sigma / 255.), size=test.shape)

On Thu, 14 Nov 2019 at 04:19, clausmichele notifications@github.com wrote:

Hi! I'm getting loads of artifacts when denoising sequences with AWGN (see a couple of examples of sequences from the DAVIS dataset below) Is this normal? Thanks

[image: image] https://user-images.githubusercontent.com/29971778/68671947-5243d380-052f-11ea-9c84-b9c4edd5e7ca.png

[image: image] https://user-images.githubusercontent.com/29971778/68671983-5f60c280-052f-11ea-945d-72aa3847a48f.png

Could you please share with me the data you tried to denoise? How did you add AWGN? I will use it for testing..

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/clausmichele/ViDeNN/issues/8?email_source=notifications&email_token=AHEVKQX44FEFONRKUL7BCA3QTT3ZJA5CNFSM4JMC4TC2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEEA3ECQ#issuecomment-553759242, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHEVKQSAFSMMXR7ZF4LVWELQTT3ZJANCNFSM4JMC4TCQ .

[clausmichele]

I guess the main difference is the clipping step which is missing. For training I used:

def gaussian_noise(sigma,image):
    gaussian = np.random.normal(0,sigma,image.shape)
    noisy_image = np.zeros(image.shape, np.float32)
    noisy_image = image + gaussian
    noisy_image = np.clip(noisy_image,0,255)
    noisy_image = noisy_image.astype(np.uint8)
    return noisy_image

[m-tassano]

OK, I understand, but if you clip the image after adding Gaussian noise, you modify the distribution of the input noise. In other words, noisy_image won't have Gaussian noise any more.

On Thu, 14 Nov 2019 at 09:38, clausmichele notifications@github.com wrote:

I guess the main difference is the clipping step which is missing. For training I used: def gaussian_noise(sigma,image): gaussian = np.random.normal(0,sigma,image.shape) noisy_image = np.zeros(image.shape, np.float32) noisy_image = image + gaussian noisy_image = np.clip(noisy_image,0,255) noisy_image = noisy_image.astype(np.uint8) return noisy_image

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/clausmichele/ViDeNN/issues/8?email_source=notifications&email_token=AHEVKQTOF2HMXF4SCDISDNDQTVBFPA5CNFSM4JMC4TC2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEEBWD4I#issuecomment-553869809, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHEVKQW4HMOWBLLETTKTG2TQTVBFPANCNFSM4JMC4TCQ .

[clausmichele]

True, but since it is not possible to have a real image where the values are higher than 255 (or 1) or lower than 0, it makes sense to me to clip the image. If I store a noisy image/video in a file instead of a local variable, they will be clipped. You could claim that AWGN is not a realistic noise type, so why bother if it can be stored in a real image. However, this project was born to treat different kind of noise and was not specificly designed for AWGN. Let me know if clipping the images solves the problem, if not I'll investigate further.

[m-tassano]

Ok, I see. Actually, in some cases you would like to denoise non clipped images, e.g. raw images or images saved as tiff. Any way, I understand your motivations for clipping images. On the other hand, your publication does compare your method to other denoising algorithms for AWGN. Bare in mind that methods such as BM3D or DnCNN haven't been designed/trained for clipped images, so their performance will certainly suffer. This makes the comparison in your publication not really fair/accurate.

In any case, thank you for your reply and your time. I will let you know if artifacts still appear even after clipping.

On Thu, 14 Nov 2019 at 09:56, clausmichele notifications@github.com wrote:

True, but since it is not possible to have a real image where the values are higher than 255 (or 1) or lower than 0, it makes sense to me to clip the image. If I store a noisy image/video in a file instead of a local variable, they will be clipped. You could claim that AWGN is not a realistic noise type, so why bother if it can be stored in a real image. However, this project was born to treat different kind of noise and was not specificly designed for AWGN. Let me know if clipping the images solves the problem, if not I'll investigate further.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/clausmichele/ViDeNN/issues/8?email_source=notifications&email_token=AHEVKQQC2WSDLYHD5Z4TQP3QTVDGTA5CNFSM4JMC4TC2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEEBXUAA#issuecomment-553875968, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHEVKQVVYA2C2IC7GM6RHA3QTVDGTANCNFSM4JMC4TCQ .