search

yu4u / noise2noise

An unofficial and partial Keras implementation of "Noise2Noise: Learning Image Restoration without Clean Data"
MIT License
1.07k stars 237 forks source link

denoising text noise #27

Open patatetom opened 5 years ago

patatetom commented 5 years ago

hello,

I tried to clean up an image enriched with text but the result is far from what I expected from the example presented : where can such a difference come from ? the fact that the fonts and colours used (especially black) are not the same as those used in training ?

thank you for sharing your work, regards, lacsaP.

$ python test_model.py --image_dir images/test/ --weight_file models/weights.057-4.796-27.68533_text_noise.hdf5 --test_noise_model clean

weights 057-4 796-27 68533_text_noise hdf5

$ python test_model.py --image_dir images/test/ --weight_file models/weights.056-4.172-28.07752_text_clean.hdf5 --test_noise_model clean

weights 056-4 172-28 07752_text_clean hdf5

savort commented 5 years ago

I have the same problem. when I input an image with text noise already add, and I used the function "get_noise_model" add text noise, It seems that the model only dealt with the text noise added , and the text noise on origin image remains unresolved. Alt Text left is the input , middle added new text noise, right is the result.

yu4u commented 5 years ago

Thank you for your reports! Oh... I'm not sure but the trained model seems seriously overfitted to the font used in training. @savort's result is very surprising. The model can distinguish the training font and the other fonts so clearly...

To address this issue, we can do several things; using multiple fonts, data augmentation with geometric transformations, and so forth. I will be busy for one or two weeks so I'll try to solve this issue after that.

savort commented 5 years ago

@yu4u 3 weeks have gone~

savort commented 5 years ago

oh , I found out the problem. If I transfer the add_noise_image to a jpeg form, when I re-read the image, the picture have been compressed and the detail of the text in the image is changed, so the model cannot denoise it.

yu4u commented 5 years ago

I did not have that idea! In order to tackle the problem, we should train model with some expected noises such as compression, resizing, and so on.

GuohongLi commented 4 years ago

@savort Would you please post out the de-noise result image after solveing the problem?