search

google / nerfies

This is the code for Deformable Neural Radiance Fields, a.k.a. Nerfies.
https://nerfies.github.io
Apache License 2.0
1.58k stars 215 forks source link

Deformation for view direction? #35

Open fishfishson opened 2 years ago

fishfishson commented 2 years ago

Hi, author!

Both nerfies and non-rigid nerf use deformation field. However, in non-rigid nerf they recompute view direction after deformation. How does nerfies deal with it? I quickly reviewed your implementation and did not find related operations. Looking forward to your reply!

longbowzhang commented 2 years ago

Similar concerns here. Quite curious why not apply deformation to the view directions? Could you explain a little bit? @keunhong Looking forward to you reply

keunhong commented 2 years ago

Hi, sorry for the late response.

This is a very good question. Our method uses observation-frame view directions which remain correct after deformation. With the appearance latent code, this provides a means to somewhat explain view dependent changes. However, you're right that since the scene geometry deforms, the reflected colors should change as well. To truly model view dependence, we'd have to consider local geometry e.g., by using normals to compute reflected ray directions. I'm sure future work will try to do this :)