search

liuyuan-pal / NeuRay

[CVPR2022] Neural Rays for Occlusion-aware Image-based Rendering
GNU General Public License v3.0
409 stars 32 forks source link

depth #20

Open caiyongqi opened 2 years ago

caiyongqi commented 2 years ago

Hi. In the DTU dataset, I found that there is a big difference between the depth map estimated using colomap (you provided) and the ground truth depth map, have you checked this?

scan1, view 0, 300x400: ground truth depth map: image depth map estimated using colomap: image

Could you check it? Thanks!!!

liuyuan-pal commented 2 years ago

Does this refer to the training set of NeuRay? As for the DTU training dataset, I adopt the processed dataset from PixelNeRF and run the colmap based on this processed version. I'm not sure what cause the difference. It may be caused by the normalization of scale and offset. Since COLMAP indeed will produce some incorrect depth in test time, it would be better to let the model receive these incorrect depth values in training.

caiyongqi commented 2 years ago

Does this refer to the training set of NeuRay? As for the DTU training dataset, I adopt the processed dataset from PixelNeRF and run the colmap based on this processed version. I'm not sure what cause the difference. It may be caused by the normalization of scale and offset. Since COLMAP indeed will produce some incorrect depth in test time, it would be better to let the model receive these incorrect depth values in training.

OK, Thanks! Could you update how to use colomap to get depth map (code or document) if you have time? I believe it will be useful.

liuyuan-pal commented 2 years ago

You may refer to the document of COLMAP. It already provides patch_match_stereo to estimate depth maps and you can read these depth maps from the provided python function like https://github.com/liuyuan-pal/NeuRay/blob/266af33b189ff6cfd7c6ae87a296a208b0219446/dataset/database.py#L129