search

autonomousvision / differentiable_volumetric_rendering

This repository contains the code for the CVPR 2020 paper "Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision"
http://www.cvlibs.net/publications/Niemeyer2020CVPR.pdf
MIT License
798 stars 90 forks source link

How to create the depthmap according to given format #16

Closed tharinduk90 closed 4 years ago

tharinduk90 commented 4 years ago

Hi! Amazing paper, and very clean code! Thanks so much for releasing this.

I am trying to use multi_view_reconstruction with depth map for 3d reconstruction .I can extract the depth map from the images taken from iphone. i got other necessary informations as well (camera params, images, masks)

depth_map deptha map from iphone

now i need to convert it(depth map - above image) to .exr format.But When i investigate the current .exr files(using affinity designer) at DTU data set there is a channel called "Y" not RGB or A.see below image,

depthmap

Q-1 Can u guide me how to convert my depth map to the format that need to use for the dvr.

Q-2 Also can u share , if there are any prepossessing code to generate the depth maps.

Q-3 Also can u share , if there are any prepossessing code to generate the masks.

m-niemeyer commented 4 years ago

Hi @tharinduk90 , thanks a lot for your interest in our project!

Q-1: You are able to change to your depth extension using this config entry. The depth maps are then loaded in this part of the data field. As you can see, we use imageio.imread for this. If you depth format cannot be read by imageio, you can also change this part of the code.

Q-2: You could probably load your depth maps with imageio, and then export them in the exr format.

Q-3: No, sorry, we did this manually. But in your case it looks like you could use your depth maps to calculate a mask (there are completely black regions, the depth value might e.g. be 0 / inf here, and then you can convert it into an appropriate binary mask).

I hope this helps a little. Good luck with your project!

tharinduk90 commented 4 years ago

Thank you for the response,

Is the depth values are in mm ?

m-niemeyer commented 4 years ago

Hi @tharinduk90 , The depth values are in line with the camera projection matrices from the DTU dataset. I never checked but I think you are right that the unit of the depth values are mm!

tharinduk90 commented 4 years ago

Hi @m-niemeyer, thank you very much, i was able to identify the issue, in my case depth values and camera projection matrices are not in same units.