search

juglab / n2v

This is the implementation of Noise2Void training.
Other
392 stars 107 forks source link

Reproducing of the paper results #41

Closed leekanggeun closed 4 years ago

leekanggeun commented 4 years ago

Hello, I have one issue for reproducing the quantitative results of paper. Even though I set up same parameter with paper, but I couldn’t get paper result for BSD68 with sigma=25 additive white gaussian noise. So Could you give me the original code to get paper result? Because I want to cite your paper but I couldn’t reproduce exactly.. I mean original code would be training and testing code for BSD400 and BSD68 include dataset.

alex-krull commented 4 years ago

Dear @leekanggeun, Due to the stochastic nature of neural network training, results can vary slightly between individual runs. So you should not expect to see the precise same numbers. However, the numbers should definitely not be systematically worse. Could you please share the numbers you obtained with us? Thanks!

leekanggeun commented 4 years ago

Thank you for response. I set epochs and iteration/epoch as like 100, 300. Accuracy was 26.92 average PSNR. I thought it converges enough to lower bound. I know it's almost impossible to reproduce the same result exactly. So could you share the avg. PSNR(numerical) for Noise2Void, as shown in Figure 7?

Thanks!!

alex-krull commented 4 years ago

This sounds definitely too low, compared to what we got. We have unfortunately currently a deadline coming up and cannot look the the problem immediately. I will get back to you in approx. a week and assemble all the data and code to reproduce the results. I hope this is still in time for your paper submission?

leekanggeun commented 4 years ago

I'd appreciate it if you would. To verify my test data set which was generated by gaussian noise(sigma 25), I used bm3d method and got 28.29 while papers bm3d resulr is 28.59. I think that random seed to generate noise will affect to final psnr. Fortunately, I got 27.14 avg PSNR by another parameter of noise2void..but it looks still a little row.. I hope assemble could be completed until November 1. Thank you!!!

tibuch commented 4 years ago

Hello @leekanggeun

Thank you for you patience.

Here is the training/validation/test data which we used in our paper and a reproducibility jupyter notebook.

Unfortunately the computation of number of blind-spots in our current release is different from the one used during our paper-experiment time. This is a bug and is fixed on n2v/fix_numPix_computation. If you want to rerun the notebook immediately you would have to install n2v from this branch.

The branch will be merged soon, but we would have to rerun all examples. Additionally we probably add the reproducibility notebook from the zip-file above. In the future reproducing the PSNR numbers should only be a jupyter notebook execution away.

tibuch commented 4 years ago

Released: https://github.com/juglab/n2v/releases/tag/v0.1.10