nerfstudio-project / nerfstudio

A collaboration friendly studio for NeRFs
https://docs.nerf.studio
Apache License 2.0
9.61k stars 1.32k forks source link

Depth map when training with nerfacto seams like not actual depth map #1558

Open VladMVLX opened 1 year ago

VladMVLX commented 1 year ago

Hello, I am working on custom export from nerfstudio based on render.py, where I need to render "rgb" and "depth" channels into separate images along camera path.

Trying to use output["depth"] image as depth map in OpenGL using perspective projection from camera intrinsics and view from extrinsic(camera path) gives very distorted result, it seams that output["depth"] is not actual depth map but rather range map. I made this assumption looking at exoporter_utils.py:158, when it used as: point = ray_bundle.origins + ray_bundle.directions * depth Is it true? Or am I missing something? If so, what`s the logic behind it?
Are there any utility functions to convert resulted range map to real depths already implemented?

decrispell commented 1 year ago

I think the comment linked below is relevant. Basically, it's just an issue of terminology.

https://github.com/nerfstudio-project/nerfstudio/issues/1028#issuecomment-1331488668

ferphy commented 7 months ago

Hello! Were you able to obtain the actual Depth Map? I'm working on a project that needs to export the distance from the camera to each pixel in the image, but I don't know how to do it.