PeterouZh / CIPS-3D

3D-aware GANs based on NeRF (arXiv).
MIT License
610 stars 60 forks source link

Question about the input of shallow nerf network #18

Closed shoutOutYangJie closed 2 years ago

shoutOutYangJie commented 2 years ago

I know nerf is a view-dependent synthesis method due to a direction input. However, in your code. I find you don't use it. Why can cips3d still work? just input the world coordinate can achieve new view synthesis? why?

PeterouZh commented 2 years ago

As stated in the original NeRF paper, the direction input helps NeRF recreate specular reflection. Without the direction, NeRF can still represent 3D scenes but may decrease the model's ability.

I don't use direction as input because I empirically found that the input direction will cause the face identity to be inconsistent under multiple views.

Although we did not input the direction, we can still obtain multi-view images by changing the camera's pose. Imagine that you are observing a stationary object that is in the world coordinate from different positions (your eyes are equivalent to a camera).