search

liuyuan-pal / NeuRay

[CVPR2022] Neural Rays for Occlusion-aware Image-based Rendering
GNU General Public License v3.0
408 stars 31 forks source link

A question about "depth_range" #6

Closed ZYY-666 closed 2 years ago

ZYY-666 commented 2 years ago

Hello, I noticed that in your database.py, DTUTrainDatabase's "depth_range" is written with a fixed value, while GoogleScannedObjectDatabase's "depth_range" is calculated based on pose. What's the difference between these two ways? And if I want to run your code on ShapeNet dataset, which way do you recommend to get depth_range?

DTUTrain： image GoogleScannedObject： image

liuyuan-pal commented 2 years ago

Hi! The depth range denotes the range to include the scene. For the DTU train dataset, the fixed depth range is used because we know such a depth range will include the whole object.

If you want to use ShapeNet, you may 1) Project the vertices into every view and compute the depth range similar to https://github.com/liuyuan-pal/NeuRay/blob/93586a07c863a58c74acfd8aebc226fc88d7af20/dataset/database.py#L519 In the ExampleDatabase, we project the sparse points from the SfM to calculate the depth range. We use np.percentile to exclude some possible outlier points and multiply 1.2 to enlarge the range to ensure the range will include the object. In your case, you may replace the points from the SfM with the points of vertices. 2) If you know the object is normalized in the unit sphere (ShapeNet models are normalized), and you know the distance from the camera to the origin, you may also manually specify the depth range according to your setting.

In my experience, using a more compact depth range leads to better performance because the searching range is smaller for the rendering to find the surface. So, option 1 is usually better than 2.

ZYY-666 commented 2 years ago

Thank you very much for your detailed and quick reply！ I will try your suggestions.