search

isl-org / Open3D

Open3D: A Modern Library for 3D Data Processing
http://www.open3d.org
Other
11.37k stars 2.29k forks source link

Camera without RGBD #4412

Closed sctrueew closed 2 years ago

sctrueew commented 2 years ago

Hi everyone,

I don't have an RGBD camera like intel realsense but I'm using a GoPro camera after the recording I extract depth with Midas(deep-learning) and save RGB and Depth frames. now I'd like to generate 3d Point cloud reconstruction like below the picture.

image

or this example: image

my result are: Frame: point3D1_rgb

Depth normalization: point3D1_depth_normlized255

Depth: point3D1_depth

How can I do that?

Thanks in advance

theNded commented 2 years ago

The reconstruction system assumes a relatively high-quality depth map from sensors. While MiDAS provides nice visualization and reasonable depth from monocular images, according to my experience it is very likely to fail in the reconstruction system.

If you want to give it a try anyways, save the depths from the RGB images in a separate depth folder and follow the instructions in the tutorial. Be aware of several caveats:

  1. MiDAS outputs inverse depth. Be sure to invert them properly.
  2. MiDAS outputs non-metric depth on most datasets. Be aware to adjust them properly and tune depth_scale factor that converts them from the uint16 pixel value to the metrical value.
  3. Use correct intrinsics.
sctrueew commented 2 years ago

@theNded Hi,

Thanks for the reply, I've put my depth frames in a folder. what example should I use? https://github.com/isl-org/Open3D/blob/master/examples/python/gui/online-processing.py Is this example works for me?

theNded commented 2 years ago

You should be looking at this tutorial.