search

liuyuan-pal / NeuRay

[CVPR2022] Neural Rays for Occlusion-aware Image-based Rendering
GNU General Public License v3.0
409 stars 32 forks source link

what's the usage of "get_diff_feats" function? #17

Open zhiwenfan opened 2 years ago

zhiwenfan commented 2 years ago

Hi Authors,

Thanks for sharing the code of the great work. Could you please explain a little bit about the usage of "get_diff_feats" function?

  1. why you inverse the near/far plane? near_inv, far_inv = -1 / near[..., None], -1 / far[..., None]
  2. why you renorm the input depth? depth_in = depth_in * (far_inv - near_inv) + near_inv depth = -1 / depth_in
  3. why you unproject the depth to point cloud and then project them back? function "project_points_ref_views"

Thanks

liuyuan-pal commented 2 years ago

Hi, thanks for your interest in our work. get_diff_feats is intended for checking the consistency of input depth maps. We project the input depth points onto neighboring views. If the color of the current view is consistent with the colors of neighboring views, then we can more confidently believe the input depth maps. If not, the input depth may be incorrect. This helps determine the variance of the occlusion probability.

About the normalization of depth values, we sample points along the ray direction in the "-1/depth" space to make it more consistent with the disparity, which is similar to the NDC space used in NeRF.