search

xmeng525 / RealTimeDenoisingNeuralBilateralGrid

[EGSR2020] Real-time Monte Carlo Denoising with the Neural Bilateral Grid
MIT License
66 stars 13 forks source link

Question about outlier removal operation #3

Closed hmfann closed 3 years ago

hmfann commented 3 years ago

Hello： Thanks for sharing your impressive work！ I have some questions about the outlier removal operation：1. What‘s the exact computation you use for outlier removal？Does it the same as [LBF15] your paper referred? 2. What’s the outlier removal execution position in your preprocessing+nn pipeline? Before or after temporal accumulation? 3. It seems that you didn’t supply outlier removed dataset, do you plan to release it later?

Thanks!

xmeng525 commented 3 years ago

Hi Hangming, 1. What‘s the exact computation you use for outlier removal？Does it the same as [LBF15] your paper referred? The algorithm is adapted from Section 4.3 in LBF15. However, we apply one modification compared to the previous work: the weights depend not only on Euclidean distance of pixels, but also on the radiometric differences. We use albedo & shading normals to test the radiometric difference. 2. What’s the outlier removal execution position in your preprocessing+nn pipeline? Before or after temporal accumulation? Before the temporal accumulation 3. It seems that you didn’t supply outlier removed dataset, do you plan to release it later? I have no plan to release the dataset, but I have released the code for outlier removal, please checkout: https://github.com/xmeng525/RealTimeDenoisingNeuralBilateralGrid/tree/master/outlier_lib Please let me know if you have any other questions!

hmfann commented 3 years ago

I will figure out the code. Thanks for your patient explanation!

hmfann commented 3 years ago

Hello, still some questions:

  1. It seems that outlier removal operation's target in outlier_unittest.py is noisy color with albedo, but the input image of data_loader.py is noisy color without albedo. Are there any modifications should be done if outlier_unittest.py's input image is changed to noisy color without albedo?
  2. Does outlier removal operation also used in BMFR-dataset? If so, should the tone_mapping manner in outlier.cu.cc be changed to gamma correction (tone_mapping() function in network.py)?
  3. Does it a casual mistake of dividing input_albedo by 255.0? Because plt.imread() returns np.array already ranging 0 to 1.

Thanks again!

xmeng525 commented 3 years ago

Hi, Hangming, 1. It seems that outlier removal operation's target in outlier_unittest.py is noisy color with albedo, but the input image of data_loader.py is noisy color without albedo. Are there any modifications should be done if outlier_unittest.py's input image is changed to noisy color without albedo? It should be the same for noisy color without albedo and noisy color with albedo.

2. Does outlier removal operation also used in BMFR-dataset? If so, should the tone_mapping manner in outlier.cu.cc be changed to gamma correction (tone_mapping() function in network.py)? I only applied outlier removal for the 64spp dataset, not the 1spp. The outlier removal approach will reduce the total energy for the whole frame. It will lead to smooth visual effect, while it will lead to lower SSIM in my experiment.

3. Does it a casual mistake of dividing input_albedo by 255.0? Because plt.imread() returns np.array already ranging 0 to 1. Thank you for pointing this out! I modified the code I uploaded so it does not rely on outside libraries. The division should not be there.

hmfann commented 3 years ago

I understand this operation now. Thank you very much, that's very kind of you!