search

autonomousvision / differentiable_volumetric_rendering

This repository contains the code for the CVPR 2020 paper "Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision"
http://www.cvlibs.net/publications/Niemeyer2020CVPR.pdf
MIT License
798 stars 90 forks source link

How to generate visual_hull_depth ? #25

Closed jay-thakur closed 4 years ago

jay-thakur commented 4 years ago

Hi,

I have downloaded NMR dataset. i have below image & respective .exr file (visual_hull_depth) 0000

I am working on my own custom dataset. here is a sample 0

Could you please guide me how to create visual_hull_depth (.exr file).

Thanks, Jay

m-niemeyer commented 4 years ago

Hi @jay-thakur , thanks a lot for your interest in the project!

The visual hull depth is only used by our model which uses only the RGB Loss: It uses this visual hull depth for the occupancy loss to encourage occupancy along rays where nothing is occupied currently, but where the GT mask suggest it should be occupied. For a more in-depth explanation, please have a look at the paper.

This means that if you use e.g. depth maps, you don't need the visual hull depth. If you want to create them, you need to a.) extract the visual hull, and b.) render it from the respective views to obtain the depth maps. If you don't want to use them during training, you can also disable the use by setting the config entry data - depth_from_visual_hull to False.

I hope this helps a little. Good luck with your research!

jay-thakur commented 4 years ago

@m-niemeyer Thank you so much.