search

google / mipnerf

Apache License 2.0
894 stars 109 forks source link

Why positional encoding for viewing direction? #32

Closed NagabhushanSN95 closed 2 years ago

NagabhushanSN95 commented 2 years ago

Positional encoding (PE) or IPE is required to allow NeRF/MipNeRF learn high frequencies. I understand that volume density and color as a function of 3d point can have discontinuities, especially at object boundaries. However, for a given 3d position, the color as a function of viewing direction is a smooth function, right? I can't imagine a scenario where it may have high frequencies. So, why do we need PE for viewing direction?

@jonbarron, I've pondered over this question for quite some time now, but didn't understand. Any intuition will be helpful.

jonbarron commented 2 years ago

Specularities are the main kind of high frequency view dependent effects. We have a follow-up paper where we explore the "right" IPE-like parameterization for view dependent effects here: https://dorverbin.github.io/refnerf/

NagabhushanSN95 commented 2 years ago

Okay. Thanks! Yes, I've come across the Ref-NeRF paper. Is there any plan to release the code for Ref-NeRF?

jonbarron commented 2 years ago

Yep, it should be out in the next couple weeks.