search

zejinwang / Blind2Unblind

This is an official implementation of the CVPR2022 paper "Blind2Unblind: Self-Supervised Image Denoising with Visible Blind Spots".
https://arxiv.org/abs/2203.06967
118 stars 13 forks source link

Model converges to identity mapping on SSID sRGB Validation Dataset #13

Closed TomHeaven closed 1 year ago

TomHeaven commented 1 year ago

Thanks for the nice paper and code.

I run the code on SSID Raw Validation dataset (ValidationNoisyBlocksRaw.mat) and it works. The generated log file (A_log_SIDD_Val.csv ) looks good as:

epoch:1,dn:35.623705/0.792260,exp:35.637747/0.792568,mid:35.633586/0.792483
epoch:2,dn:36.691027/0.813433,exp:36.729187/0.814111,mid:36.717283/0.813909
epoch:3,dn:37.662347/0.819666,exp:37.696882/0.820399,mid:37.686802/0.820189
epoch:4,dn:36.591700/0.809505,exp:36.619088/0.810321,mid:36.611489/0.810091
epoch:5,dn:36.354757/0.793341,exp:36.395038/0.794624,mid:36.384245/0.794274
epoch:6,dn:40.262343/0.923484,exp:40.369246/0.924950,mid:40.342693/0.924603
epoch:7,dn:41.634836/0.939280,exp:41.776404/0.940960,mid:41.743518/0.940571
epoch:8,dn:41.397646/0.937913,exp:41.548824/0.939674,mid:41.512758/0.939290
epoch:9,dn:41.689088/0.954902,exp:41.854164/0.956958,mid:41.821227/0.956529
epoch:10,dn:44.555741/0.968270,exp:44.853289/0.970512,mid:44.796125/0.970074
epoch:11,dn:44.999689/0.969440,exp:45.281381/0.971816,mid:45.235513/0.971367
epoch:12,dn:45.419838/0.971243,exp:45.743328/0.973611,mid:45.686149/0.973167
epoch:13,dn:44.878654/0.968614,exp:45.177972/0.971009,mid:45.125890/0.970557
epoch:14,dn:45.335208/0.970630,exp:45.648264/0.973065,mid:45.598885/0.972619
epoch:15,dn:45.373832/0.972728,exp:45.676182/0.975205,mid:45.633587/0.974751
epoch:16,dn:45.089008/0.973235,exp:45.430240/0.975746,mid:45.374611/0.975286
epoch:17,dn:44.889191/0.974528,exp:45.266475/0.977052,mid:45.193553/0.976604
epoch:18,dn:43.930426/0.971998,exp:44.150318/0.974647,mid:44.124846/0.974177
epoch:19,dn:42.528658/0.962436,exp:42.753665/0.965103,mid:42.718973/0.964630
epoch:20,dn:46.702977/0.979201,exp:47.145411/0.981932,mid:47.081676/0.981467
...
epoch:97,dn:48.916775/0.987458,exp:49.452608/0.989204,mid:49.453884/0.989185
epoch:98,dn:49.147523/0.987632,exp:49.733000/0.989374,mid:49.732428/0.989355
epoch:99,dn:49.113910/0.987588,exp:49.683852/0.989348,mid:49.684658/0.989331
epoch:100,dn:49.116245/0.987665,exp:49.708167/0.989422,mid:49.707363/0.989404

However, When I run the code (with slight modifications to fit the three-channel input) on SSID sRGB Validation dataset (ValidationNoisyBlocksSrgb.mat). The generated log file (A_log_SIDD_Val.csv ) shows that the model seems to converge to the identity mapping.

epoch:1,dn:27.404763/0.838748,exp:27.444009/0.842822,mid:27.433564/0.842193
epoch:2,dn:29.671704/0.784196,exp:29.695574/0.784053,mid:29.700154/0.785936
epoch:3,dn:28.303753/0.660923,exp:28.070521/0.650066,mid:28.177892/0.656177
epoch:4,dn:27.542834/0.609191,exp:27.170839/0.590442,mid:27.331493/0.599662
epoch:5,dn:27.040710/0.555477,exp:26.471492/0.526616,mid:26.702903/0.539143
epoch:6,dn:26.566351/0.510125,exp:25.901059/0.477941,mid:26.156954/0.490571
epoch:7,dn:26.041377/0.476580,exp:25.474742/0.451214,mid:25.682107/0.460485
epoch:8,dn:25.769698/0.445767,exp:25.371750/0.432081,mid:25.514558/0.437149
epoch:9,dn:25.587256/0.432471,exp:25.326529/0.426420,mid:25.425609/0.429413
epoch:10,dn:25.381982/0.417433,exp:25.222243/0.416812,mid:25.296105/0.418925
epoch:11,dn:25.121333/0.406872,exp:25.049778/0.410772,mid:25.104690/0.412451
epoch:12,dn:25.027494/0.394207,exp:25.004860/0.401712,mid:25.056882/0.403242
epoch:13,dn:25.063673/0.393456,exp:25.086481/0.404311,mid:25.133990/0.405482
epoch:14,dn:24.943280/0.385651,exp:25.029813/0.400656,mid:25.066898/0.401026
epoch:15,dn:24.704481/0.381718,exp:24.856096/0.400906,mid:24.876091/0.400216
epoch:16,dn:24.606720/0.371080,exp:24.814270/0.391729,mid:24.826118/0.391072
epoch:17,dn:24.608155/0.370326,exp:24.833266/0.393604,mid:24.845978/0.392103
epoch:18,dn:24.484653/0.362355,exp:24.762162/0.389126,mid:24.774272/0.387294
...

A visual result of SIDD_Val_023-027-005_mid looks like this: SIDD_Val_023-027-005_mid

Have you tried the code on SSID sRGB Validation dataset? Any suggestions are appreciated.

zejinwang commented 1 year ago

This work employs masked denoising as an auxiliary task, making it suitable for pixel-wise independent noise. The noise in the raw domain of the SIDD dataset conforms to this requirement. However, in the sRGB domain, demosaicing and filtering operations often degrade the spatial independence of noise in the raw domain, introducing spatially correlated structured noise. Therefore, it is recommended to use Blind2Unblind in the raw domain rather than the sRGB domain.

TomHeaven commented 1 year ago

Thanks for the explanation.